2024-09-16T04:34:40.842 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T04:34:40.859 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T04:34:40.941 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:00:02-rados-squid-distro-default-smithi/7906051 branch: squid description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906051' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-15_21:00:02-rados-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: global: mon election default strategy: 3 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) mon_bind_msgr2: false sha1: 8921c8ef947807b717db041d85f0761b7914d5fc ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 700 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 seed: 5616 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 15927/100000 suite: rados suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCJW9FBA3NDU5n9ipFsaafiiQByl+wKwFnjlgvHWvjSfnEH6m9uAJTiriqemXBzceYOjBn5okI12ySN4wGNeW+A= tasks: - install: null - ceph: log-ignorelist: - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(SLOW_OPS\) - slow request - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - rep_lost_unfound_delete: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:00:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T04:34:40.941 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-16T04:34:40.942 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-16T04:34:40.942 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T04:34:40.944 INFO:teuthology.task.internal:Checking packages... 2024-09-16T04:34:40.965 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-16T04:34:40.965 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T04:34:40.966 INFO:teuthology.packaging:ref: None 2024-09-16T04:34:40.966 INFO:teuthology.packaging:tag: None 2024-09-16T04:34:40.966 INFO:teuthology.packaging:branch: squid 2024-09-16T04:34:40.966 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T04:34:40.966 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-16T04:34:41.167 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-326-g8921c8ef-1jammy 2024-09-16T04:34:41.169 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T04:34:41.175 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T04:34:41.176 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T04:34:41.217 INFO:teuthology.task.internal:Saving configuration 2024-09-16T04:34:41.229 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T04:34:41.237 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T04:34:41.263 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:00:02-rados-squid-distro-default-smithi/7906051', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 04:29:32.863656', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOAffF8lYtkQos7YYCCV0/bNTEfgjdXm0wvmq+8leto/'} 2024-09-16T04:34:41.263 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T04:34:41.270 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2024-09-16T04:34:41.270 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T04:34:41.305 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa102886e60>, signals=[15]) 2024-09-16T04:34:41.305 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T04:34:41.319 INFO:teuthology.task.internal:Opening connections... 2024-09-16T04:34:41.319 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-09-16T04:34:41.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T04:34:41.412 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T04:34:41.418 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-09-16T04:34:41.538 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-09-16T04:34:41.538 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-09-16T04:34:41.583 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-16T04:34:41.583 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Ubuntu" 2024-09-16T04:34:41.583 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="22.04" 2024-09-16T04:34:41.583 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-16T04:34:41.583 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_CODENAME=jammy 2024-09-16T04:34:41.583 INFO:teuthology.orchestra.run.smithi012.stdout:ID=ubuntu 2024-09-16T04:34:41.584 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE=debian 2024-09-16T04:34:41.584 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-16T04:34:41.584 INFO:teuthology.orchestra.run.smithi012.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-16T04:34:41.584 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-16T04:34:41.584 INFO:teuthology.orchestra.run.smithi012.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-16T04:34:41.584 INFO:teuthology.orchestra.run.smithi012.stdout:UBUNTU_CODENAME=jammy 2024-09-16T04:34:41.584 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-09-16T04:34:41.601 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T04:34:41.609 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T04:34:41.663 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T04:34:41.663 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-09-16T04:34:41.668 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T04:34:41.673 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T04:34:41.674 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-09-16T04:34:41.734 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T04:34:41.767 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T04:34:41.803 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T04:34:41.803 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T04:34:41.804 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T04:34:41.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:34:41.809 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-09-16T04:34:41.856 INFO:teuthology.orchestra.run.smithi012.stdout:5.15.0-56-generic 2024-09-16T04:34:41.856 INFO:teuthology.task.kernel:Running kernel on smithi012: 5.15.0-56-generic 2024-09-16T04:34:41.856 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2024-09-16T04:34:42.009 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-09-16T04:34:42.148 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-16T04:34:42.259 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T04:34:42.345 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-16T04:34:42.362 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-16T04:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-16T04:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-16T04:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-16T04:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-16T04:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-16T04:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-16T04:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-16T04:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-16T04:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-16T04:34:42.510 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-16T04:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-16T04:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-16T04:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-16T04:34:42.522 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-16T04:34:42.522 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-16T04:34:42.697 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-16T04:34:42.835 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-16T04:34:43.029 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-16T04:34:43.188 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-16T04:34:43.202 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-16T04:34:43.203 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-16T04:34:43.205 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-16T04:34:43.273 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-16T04:34:43.289 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-16T04:34:43.290 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-16T04:34:43.312 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-16T04:34:43.340 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-16T04:34:43.345 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-16T04:34:43.353 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-16T04:34:43.353 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-16T04:34:43.355 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-16T04:34:43.428 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-16T04:34:43.429 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-16T04:34:43.430 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-16T04:34:43.432 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-16T04:34:43.432 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-16T04:34:43.434 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-16T04:34:43.435 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-16T04:34:43.515 INFO:teuthology.orchestra.run.smithi012.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-16T04:34:43.516 INFO:teuthology.orchestra.run.smithi012.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-16T04:34:49.363 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 16.0 MB in 2s (7,315 kB/s) 2024-09-16T04:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-16T04:34:50.881 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T04:34:50.936 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-16T04:34:51.133 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-09-16T04:34:51.134 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-09-16T04:34:51.310 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T04:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T04:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T04:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T04:34:51.312 INFO:teuthology.orchestra.run.smithi012.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-16T04:34:51.312 INFO:teuthology.orchestra.run.smithi012.stdout: python2.7-minimal 2024-09-16T04:34:51.312 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T04:34:51.313 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2024-09-16T04:34:51.313 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T04:34:51.313 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-16T04:34:51.313 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-16T04:34:51.314 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2024-09-16T04:34:51.315 INFO:teuthology.orchestra.run.smithi012.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-16T04:34:51.315 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2024-09-16T04:34:51.315 INFO:teuthology.orchestra.run.smithi012.stdout: thermald 2024-09-16T04:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2024-09-16T04:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T04:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-16T04:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-16T04:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages will be upgraded: 2024-09-16T04:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-16T04:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T04:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 113 MB of archives. 2024-09-16T04:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-16T04:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-16T04:34:51.832 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-16T04:34:51.929 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-16T04:34:52.792 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-16T04:34:52.792 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-16T04:34:52.792 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-16T04:34:52.923 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-16T04:34:52.944 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-16T04:34:53.351 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 113 MB in 2s (72.2 MB/s) 2024-09-16T04:34:53.549 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-16T04:34:55.668 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-16T04:34:55.672 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T04:34:56.095 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T04:34:58.214 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-16T04:34:58.236 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T04:34:58.342 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T04:34:58.842 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-16T04:34:58.863 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T04:34:58.900 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T04:35:05.611 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T04:35:05.753 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T04:35:06.049 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T04:35:06.206 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T04:35:06.501 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-16T04:35:06.524 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-16T04:35:06.584 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T04:35:12.816 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-16T04:35:12.851 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T04:35:12.888 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T04:35:15.759 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T04:35:15.892 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T04:35:16.251 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T04:35:16.372 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T04:35:16.523 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-16T04:35:16.651 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T04:35:18.083 INFO:teuthology.orchestra.run.smithi012.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-16T04:35:18.084 INFO:teuthology.orchestra.run.smithi012.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-16T04:35:18.084 INFO:teuthology.orchestra.run.smithi012.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-16T04:35:18.084 INFO:teuthology.orchestra.run.smithi012.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-16T04:35:18.646 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T04:35:20.046 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T04:35:21.500 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-16T04:35:21.632 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-16T04:35:21.758 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T04:35:21.820 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-16T04:35:21.820 INFO:teuthology.orchestra.run.smithi012.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-16T04:35:36.218 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-16T04:35:36.219 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub' 2024-09-16T04:35:36.231 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T04:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout:Generating grub configuration file ... 2024-09-16T04:35:36.581 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T04:35:36.607 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T04:35:36.947 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T04:35:36.952 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T04:35:37.018 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T04:35:37.024 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T04:35:37.188 INFO:teuthology.orchestra.run.smithi012.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T04:35:37.188 INFO:teuthology.orchestra.run.smithi012.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T04:35:37.188 INFO:teuthology.orchestra.run.smithi012.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T04:35:37.227 INFO:teuthology.orchestra.run.smithi012.stdout:done 2024-09-16T04:35:37.731 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2024-09-16T04:35:37.763 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2024-09-16T04:35:37.763 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2024-09-16T04:35:37.763 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2024-09-16T04:35:37.764 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2024-09-16T04:35:37.764 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 21 2024-09-16T04:35:37.764 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T04:35:37.764 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2024-09-16T04:35:37.764 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2024-09-16T04:35:37.764 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.15.0.121.121 2024-09-16T04:35:37.764 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T04:35:37.764 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T04:35:37.764 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2024-09-16T04:35:37.764 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2024-09-16T04:35:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2024-09-16T04:35:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2024-09-16T04:35:37.765 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-16T04:35:37.765 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T04:35:37.765 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T04:35:37.765 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T04:35:37.765 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T04:35:37.765 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2024-09-16T04:35:37.958 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-09-16T04:35:38.073 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T04:35:38.082 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-16T04:35:38.099 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T04:35:38.193 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-16T04:35:39.514 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-16T04:35:39.534 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T04:35:39.588 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-16T04:35:39.779 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-09-16T04:35:39.780 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-09-16T04:35:39.952 INFO:teuthology.orchestra.run.smithi012.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-16T04:35:39.952 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T04:35:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T04:35:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T04:35:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T04:35:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T04:35:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T04:35:39.953 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T04:35:39.993 INFO:teuthology.orchestra.run.smithi012.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T04:35:39.995 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2024-09-16T04:35:40.058 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2024-09-16T04:35:40.059 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2024-09-16T04:35:40.059 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2024-09-16T04:35:40.059 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2024-09-16T04:35:40.059 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 21 2024-09-16T04:35:40.059 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T04:35:40.059 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2024-09-16T04:35:40.059 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2024-09-16T04:35:40.059 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.15.0.121.121 2024-09-16T04:35:40.059 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T04:35:40.059 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T04:35:40.059 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2024-09-16T04:35:40.059 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2024-09-16T04:35:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2024-09-16T04:35:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2024-09-16T04:35:40.060 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-09-16T04:35:40.107 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.2iCTXrSlJY 2024-09-16T04:35:40.108 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub/grub.cfg /tmp/tmp.2iCTXrSlJY 2024-09-16T04:35:40.164 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.2iCTXrSlJY 2024-09-16T04:35:40.287 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.2iCTXrSlJY is 10KB 2024-09-16T04:35:40.299 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.2iCTXrSlJY 2024-09-16T04:35:40.305 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-16T04:35:40.360 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:35:40.360 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-16T04:35:40.360 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-16T04:35:40.427 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-16T04:35:40.427 DEBUG:teuthology.orchestra.run.smithi012:> sudo update-grub 2024-09-16T04:35:41.806 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub' 2024-09-16T04:35:41.806 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T04:35:41.810 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2024-09-16T04:35:41.999 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T04:35:42.005 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T04:35:42.268 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T04:35:42.273 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T04:35:42.316 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T04:35:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T04:35:42.447 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T04:35:42.447 INFO:teuthology.orchestra.run.smithi012.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T04:35:42.447 INFO:teuthology.orchestra.run.smithi012.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T04:35:42.466 INFO:teuthology.orchestra.run.smithi012.stderr:done 2024-09-16T04:35:42.475 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2024-09-16T04:36:12.505 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T04:36:12.506 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-16T04:36:12.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T04:36:30.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-09-16T04:36:39.917 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-16T04:36:39.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T04:36:58.301 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-09-16T04:37:10.314 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-16T04:37:10.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T04:37:12.807 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-09-16T04:37:27.821 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-16T04:37:27.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T04:37:28.051 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-09-16T04:37:28.709 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-16T04:37:28.709 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-16T04:37:28.709 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-09-16T04:37:28.758 INFO:teuthology.orchestra.run.smithi012.stdout:5.15.0-121-generic 2024-09-16T04:37:28.758 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-16T04:37:28.758 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T04:37:28.758 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-16T04:37:29.759 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T04:37:29.759 DEBUG:teuthology.orchestra.run.smithi012:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T04:37:29.834 INFO:teuthology.orchestra.run.smithi012.stdout:ttyS1 2024-09-16T04:37:29.851 DEBUG:teuthology.parallel:result is None 2024-09-16T04:37:29.851 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T04:37:29.861 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T04:37:29.862 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T04:37:29.868 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T04:37:29.873 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T04:37:29.883 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T04:37:29.883 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T04:37:29.927 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T04:37:29.935 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T04:37:29.935 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T04:37:29.988 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T04:37:29.999 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T04:37:30.001 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T04:37:30.061 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T04:37:30.062 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T04:37:30.075 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T04:37:30.113 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T04:37:30.113 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T04:37:30.121 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T04:37:30.170 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T04:37:30.221 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:37:30.221 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T04:37:30.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-09-16T04:37:30.365 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T04:37:30.373 INFO:teuthology.task.internal:Starting timer... 2024-09-16T04:37:30.373 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T04:37:30.383 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T04:37:30.392 DEBUG:teuthology.task.selinux:Excluding smithi012: OS 'ubuntu' does not support SELinux 2024-09-16T04:37:30.392 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T04:37:30.392 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-16T04:37:30.392 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T04:37:30.392 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T04:37:30.401 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-16T04:37:30.404 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-16T04:37:30.516 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T04:37:30.527 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T04:37:30.529 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T04:45:11.079 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com')] 2024-09-16T04:45:11.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-16T04:45:11.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T04:45:11.167 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-09-16T04:45:11.251 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-16T04:45:11.251 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T04:45:11.264 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T04:45:11.264 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T04:45:11.265 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T04:45:11.318 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-16T04:45:11.318 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: Command line: ntpd -gq 2024-09-16T04:45:11.318 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: ---------------------------------------------------- 2024-09-16T04:45:11.318 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: ntp-4 is maintained by Network Time Foundation, 2024-09-16T04:45:11.318 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-16T04:45:11.318 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: corporation. Support and training for ntp-4 are 2024-09-16T04:45:11.318 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: available at https://www.nwtime.org/support 2024-09-16T04:45:11.318 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: ---------------------------------------------------- 2024-09-16T04:45:11.319 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: proto: precision = 0.046 usec (-24) 2024-09-16T04:45:11.319 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: basedate set to 2022-02-04 2024-09-16T04:45:11.319 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: gps base set to 2022-02-06 (week 2196) 2024-09-16T04:45:11.319 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T04:45:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T04:45:11.320 INFO:teuthology.orchestra.run.smithi012.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-16T04:45:11.321 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: restrict ::: KOD does nothing without LIMITED. 2024-09-16T04:45:11.321 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: Listen and drop on 0 v6wildcard [::]:123 2024-09-16T04:45:11.321 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-16T04:45:11.321 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: Listen normally on 2 lo 127.0.0.1:123 2024-09-16T04:45:11.321 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: Listen normally on 3 ens1f0 172.21.15.12:123 2024-09-16T04:45:11.321 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: Listen normally on 4 lo [::1]:123 2024-09-16T04:45:11.321 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:25ea%3]:123 2024-09-16T04:45:11.321 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:11 ntpd[18395]: Listening on routing socket on fd #22 for interface updates 2024-09-16T04:45:12.318 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:12 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:12.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:12 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:13.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:13 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:13.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:13 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:13.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:13 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:13.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:13 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:13.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:13 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T04:45:13.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:13 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T04:45:15.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:15 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:15 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:15 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:15 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:15 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:15 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:15 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T04:45:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:15 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T04:45:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:15 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:15 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:15.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:15 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T04:45:15.321 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:15 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T04:45:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:16 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:16 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:16 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:16 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:16 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T04:45:16.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:16 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T04:45:17.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:17 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:17.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:17 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:17.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:17 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:17.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:17 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:17.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:17 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:17.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:17 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:17.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:17 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T04:45:17.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:17 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T04:45:17.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:17 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:17.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:17 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:17.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:17 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T04:45:17.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:17 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T04:45:18.318 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:18 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:18.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:18 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:18.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:18 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:18.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:18 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:18.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:18 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T04:45:18.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:18 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T04:45:19.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:19.319 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T04:45:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T04:45:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T04:45:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T04:45:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T04:45:19.320 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T04:45:19.321 INFO:teuthology.orchestra.run.smithi012.stdout:16 Sep 04:45:19 ntpd[18395]: ntpd: time slew -0.001202 s 2024-09-16T04:45:19.321 INFO:teuthology.orchestra.run.smithi012.stdout:ntpd: time slew -0.001202s 2024-09-16T04:45:19.321 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-16T04:45:19.321 INFO:teuthology.orchestra.run.smithi012.stderr:16 Sep 04:45:19 ntpd[18395]: can't open /var/log/ntpstats/loopstats.20240916: Permission denied 2024-09-16T04:45:19.388 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T04:45:19.388 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2024-09-16T04:45:19.388 INFO:teuthology.orchestra.run.smithi012.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T04:45:19.388 INFO:teuthology.orchestra.run.smithi012.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T04:45:19.388 INFO:teuthology.orchestra.run.smithi012.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T04:45:19.389 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T04:45:19.390 INFO:teuthology.run_tasks:Running task install... 2024-09-16T04:45:19.398 DEBUG:teuthology.task.install:project ceph 2024-09-16T04:45:19.398 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-16T04:45:19.399 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-16T04:45:19.399 INFO:teuthology.task.install:Using flavor: default 2024-09-16T04:45:19.404 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T04:45:19.404 INFO:teuthology.task.install:extra packages: [] 2024-09-16T04:45:19.405 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-key list | grep Ceph 2024-09-16T04:45:19.486 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-16T04:45:19.516 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-16T04:45:19.516 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph.com (release key) 2024-09-16T04:45:19.517 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-16T04:45:19.517 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T04:45:19.661 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default/ 2024-09-16T04:45:19.661 INFO:teuthology.task.install.deb:Package version is 19.1.1-326-g8921c8ef-1jammy 2024-09-16T04:45:19.770 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:45:19.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-16T04:45:19.785 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-09-16T04:45:19.956 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-16T04:45:19.956 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T04:45:19.967 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-16T04:45:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T04:45:20.012 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy InRelease 2024-09-16T04:45:20.033 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-16T04:45:20.055 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-16T04:45:20.076 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,422 B] 2024-09-16T04:45:20.108 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-16T04:45:20.320 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 29.0 kB in 0s (83.9 kB/s) 2024-09-16T04:45:21.350 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-16T04:45:21.367 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-326-g8921c8ef-1jammy cephadm=19.1.1-326-g8921c8ef-1jammy ceph-mds=19.1.1-326-g8921c8ef-1jammy ceph-mgr=19.1.1-326-g8921c8ef-1jammy ceph-common=19.1.1-326-g8921c8ef-1jammy ceph-fuse=19.1.1-326-g8921c8ef-1jammy ceph-test=19.1.1-326-g8921c8ef-1jammy ceph-volume=19.1.1-326-g8921c8ef-1jammy radosgw=19.1.1-326-g8921c8ef-1jammy python3-rados=19.1.1-326-g8921c8ef-1jammy python3-rgw=19.1.1-326-g8921c8ef-1jammy python3-cephfs=19.1.1-326-g8921c8ef-1jammy python3-rbd=19.1.1-326-g8921c8ef-1jammy libcephfs2=19.1.1-326-g8921c8ef-1jammy libcephfs-dev=19.1.1-326-g8921c8ef-1jammy librados2=19.1.1-326-g8921c8ef-1jammy librbd1=19.1.1-326-g8921c8ef-1jammy rbd-fuse=19.1.1-326-g8921c8ef-1jammy 2024-09-16T04:45:21.431 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-16T04:45:21.646 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-09-16T04:45:21.647 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-09-16T04:45:21.869 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T04:45:21.869 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T04:45:21.869 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T04:45:21.869 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T04:45:21.869 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T04:45:21.869 INFO:teuthology.orchestra.run.smithi012.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T04:45:21.870 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T04:45:21.870 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2024-09-16T04:45:21.870 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-16T04:45:21.871 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-16T04:45:21.871 INFO:teuthology.orchestra.run.smithi012.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-16T04:45:21.871 INFO:teuthology.orchestra.run.smithi012.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-16T04:45:21.872 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-16T04:45:21.872 INFO:teuthology.orchestra.run.smithi012.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-16T04:45:21.872 INFO:teuthology.orchestra.run.smithi012.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-16T04:45:21.873 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-16T04:45:21.873 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-16T04:45:21.873 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-16T04:45:21.873 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-16T04:45:21.873 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-16T04:45:21.873 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-16T04:45:21.873 INFO:teuthology.orchestra.run.smithi012.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-16T04:45:21.873 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-16T04:45:21.873 INFO:teuthology.orchestra.run.smithi012.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T04:45:21.873 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-16T04:45:21.873 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-16T04:45:21.873 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T04:45:21.873 INFO:teuthology.orchestra.run.smithi012.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-16T04:45:21.874 INFO:teuthology.orchestra.run.smithi012.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-16T04:45:21.874 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-16T04:45:21.874 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-16T04:45:21.874 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet zip 2024-09-16T04:45:21.874 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2024-09-16T04:45:21.874 INFO:teuthology.orchestra.run.smithi012.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-16T04:45:21.874 INFO:teuthology.orchestra.run.smithi012.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-16T04:45:21.874 INFO:teuthology.orchestra.run.smithi012.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-16T04:45:21.874 INFO:teuthology.orchestra.run.smithi012.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-16T04:45:21.874 INFO:teuthology.orchestra.run.smithi012.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-16T04:45:21.875 INFO:teuthology.orchestra.run.smithi012.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-16T04:45:21.875 INFO:teuthology.orchestra.run.smithi012.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-16T04:45:21.875 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2024-09-16T04:45:21.875 INFO:teuthology.orchestra.run.smithi012.stdout: btrfs-tools 2024-09-16T04:45:21.985 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2024-09-16T04:45:21.985 INFO:teuthology.orchestra.run.smithi012.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-16T04:45:21.985 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-16T04:45:21.985 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-16T04:45:21.985 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-16T04:45:21.985 INFO:teuthology.orchestra.run.smithi012.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-16T04:45:21.985 INFO:teuthology.orchestra.run.smithi012.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-16T04:45:21.986 INFO:teuthology.orchestra.run.smithi012.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-16T04:45:21.986 INFO:teuthology.orchestra.run.smithi012.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-16T04:45:21.987 INFO:teuthology.orchestra.run.smithi012.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-16T04:45:21.987 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-16T04:45:21.987 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-16T04:45:21.987 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-16T04:45:21.987 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-16T04:45:21.987 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-16T04:45:21.987 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-16T04:45:21.987 INFO:teuthology.orchestra.run.smithi012.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-16T04:45:21.987 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-16T04:45:21.987 INFO:teuthology.orchestra.run.smithi012.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T04:45:21.987 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-16T04:45:21.987 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-16T04:45:21.987 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-16T04:45:21.988 INFO:teuthology.orchestra.run.smithi012.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-16T04:45:21.988 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-16T04:45:21.988 INFO:teuthology.orchestra.run.smithi012.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-16T04:45:21.988 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-16T04:45:21.988 INFO:teuthology.orchestra.run.smithi012.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-16T04:45:22.131 INFO:teuthology.orchestra.run.smithi012.stdout:0 upgraded, 118 newly installed, 0 to remove and 331 not upgraded. 2024-09-16T04:45:22.131 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 181 MB of archives. 2024-09-16T04:45:22.132 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 829 MB of additional disk space will be used. 2024-09-16T04:45:22.132 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-326-g8921c8ef-1jammy [3,590 kB] 2024-09-16T04:45:22.174 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-16T04:45:22.428 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-326-g8921c8ef-1jammy [3,241 kB] 2024-09-16T04:45:22.558 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-326-g8921c8ef-1jammy [719 kB] 2024-09-16T04:45:22.581 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-326-g8921c8ef-1jammy [356 kB] 2024-09-16T04:45:22.594 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-16T04:45:22.597 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-326-g8921c8ef-1jammy [32.8 kB] 2024-09-16T04:45:22.598 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-326-g8921c8ef-1jammy [180 kB] 2024-09-16T04:45:22.602 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-326-g8921c8ef-1jammy [67.8 kB] 2024-09-16T04:45:22.604 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-326-g8921c8ef-1jammy [333 kB] 2024-09-16T04:45:22.606 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-16T04:45:22.616 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-326-g8921c8ef-1jammy [6,875 kB] 2024-09-16T04:45:22.687 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-16T04:45:22.792 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-326-g8921c8ef-1jammy [112 kB] 2024-09-16T04:45:22.794 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-326-g8921c8ef-1jammy [472 kB] 2024-09-16T04:45:22.802 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-326-g8921c8ef-1jammy [25.9 MB] 2024-09-16T04:45:22.916 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-16T04:45:22.926 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-16T04:45:22.959 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-16T04:45:22.967 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-16T04:45:22.968 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-16T04:45:22.970 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-16T04:45:22.987 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-16T04:45:22.992 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-16T04:45:22.996 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-16T04:45:22.998 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-16T04:45:23.079 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-16T04:45:23.083 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-16T04:45:23.088 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-16T04:45:23.091 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-16T04:45:23.092 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-16T04:45:23.093 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-16T04:45:23.095 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-16T04:45:23.160 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-16T04:45:23.161 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-16T04:45:23.162 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-16T04:45:23.242 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-16T04:45:23.261 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-16T04:45:23.262 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-16T04:45:23.362 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-16T04:45:23.363 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-326-g8921c8ef-1jammy [5,145 kB] 2024-09-16T04:45:23.363 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-16T04:45:23.364 INFO:teuthology.orchestra.run.smithi012.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-16T04:45:23.365 INFO:teuthology.orchestra.run.smithi012.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-16T04:45:23.367 INFO:teuthology.orchestra.run.smithi012.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-16T04:45:23.368 INFO:teuthology.orchestra.run.smithi012.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-16T04:45:23.369 INFO:teuthology.orchestra.run.smithi012.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-16T04:45:23.404 INFO:teuthology.orchestra.run.smithi012.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-16T04:45:23.405 INFO:teuthology.orchestra.run.smithi012.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-16T04:45:23.427 INFO:teuthology.orchestra.run.smithi012.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-326-g8921c8ef-1jammy [245 kB] 2024-09-16T04:45:23.429 INFO:teuthology.orchestra.run.smithi012.stdout:Get:51 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-326-g8921c8ef-1jammy [124 kB] 2024-09-16T04:45:23.431 INFO:teuthology.orchestra.run.smithi012.stdout:Get:52 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-326-g8921c8ef-1jammy [1,555 kB] 2024-09-16T04:45:23.445 INFO:teuthology.orchestra.run.smithi012.stdout:Get:53 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-326-g8921c8ef-1jammy [6,143 kB] 2024-09-16T04:45:23.485 INFO:teuthology.orchestra.run.smithi012.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-16T04:45:23.496 INFO:teuthology.orchestra.run.smithi012.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-16T04:45:23.496 INFO:teuthology.orchestra.run.smithi012.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-16T04:45:23.497 INFO:teuthology.orchestra.run.smithi012.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-16T04:45:23.498 INFO:teuthology.orchestra.run.smithi012.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-16T04:45:23.501 INFO:teuthology.orchestra.run.smithi012.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-16T04:45:23.505 INFO:teuthology.orchestra.run.smithi012.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-16T04:45:23.507 INFO:teuthology.orchestra.run.smithi012.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-16T04:45:23.534 INFO:teuthology.orchestra.run.smithi012.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-326-g8921c8ef-1jammy [22.8 MB] 2024-09-16T04:45:23.566 INFO:teuthology.orchestra.run.smithi012.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-16T04:45:23.568 INFO:teuthology.orchestra.run.smithi012.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-16T04:45:23.648 INFO:teuthology.orchestra.run.smithi012.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-16T04:45:23.649 INFO:teuthology.orchestra.run.smithi012.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-16T04:45:23.650 INFO:teuthology.orchestra.run.smithi012.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-16T04:45:23.655 INFO:teuthology.orchestra.run.smithi012.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-16T04:45:23.818 INFO:teuthology.orchestra.run.smithi012.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-16T04:45:23.819 INFO:teuthology.orchestra.run.smithi012.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-16T04:45:23.822 INFO:teuthology.orchestra.run.smithi012.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-16T04:45:23.823 INFO:teuthology.orchestra.run.smithi012.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-16T04:45:23.825 INFO:teuthology.orchestra.run.smithi012.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-16T04:45:23.862 INFO:teuthology.orchestra.run.smithi012.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-16T04:45:23.865 INFO:teuthology.orchestra.run.smithi012.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-16T04:45:23.866 INFO:teuthology.orchestra.run.smithi012.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-16T04:45:23.987 INFO:teuthology.orchestra.run.smithi012.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-16T04:45:23.988 INFO:teuthology.orchestra.run.smithi012.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-16T04:45:23.988 INFO:teuthology.orchestra.run.smithi012.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-326-g8921c8ef-1jammy [14.1 kB] 2024-09-16T04:45:23.988 INFO:teuthology.orchestra.run.smithi012.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-326-g8921c8ef-1jammy [877 kB] 2024-09-16T04:45:23.988 INFO:teuthology.orchestra.run.smithi012.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-16T04:45:24.047 INFO:teuthology.orchestra.run.smithi012.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-326-g8921c8ef-1jammy [2,462 kB] 2024-09-16T04:45:24.047 INFO:teuthology.orchestra.run.smithi012.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-16T04:45:24.048 INFO:teuthology.orchestra.run.smithi012.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-16T04:45:24.050 INFO:teuthology.orchestra.run.smithi012.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-16T04:45:24.050 INFO:teuthology.orchestra.run.smithi012.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-16T04:45:24.052 INFO:teuthology.orchestra.run.smithi012.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-16T04:45:24.060 INFO:teuthology.orchestra.run.smithi012.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-16T04:45:24.069 INFO:teuthology.orchestra.run.smithi012.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-16T04:45:24.077 INFO:teuthology.orchestra.run.smithi012.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-326-g8921c8ef-1jammy [794 kB] 2024-09-16T04:45:24.088 INFO:teuthology.orchestra.run.smithi012.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-326-g8921c8ef-1jammy [152 kB] 2024-09-16T04:45:24.090 INFO:teuthology.orchestra.run.smithi012.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-326-g8921c8ef-1jammy [2,275 kB] 2024-09-16T04:45:24.114 INFO:teuthology.orchestra.run.smithi012.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-326-g8921c8ef-1jammy [8,625 kB] 2024-09-16T04:45:24.148 INFO:teuthology.orchestra.run.smithi012.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-16T04:45:24.148 INFO:teuthology.orchestra.run.smithi012.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-16T04:45:24.156 INFO:teuthology.orchestra.run.smithi012.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-16T04:45:24.163 INFO:teuthology.orchestra.run.smithi012.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-16T04:45:24.165 INFO:teuthology.orchestra.run.smithi012.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-16T04:45:24.167 INFO:teuthology.orchestra.run.smithi012.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-16T04:45:24.178 INFO:teuthology.orchestra.run.smithi012.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-16T04:45:24.179 INFO:teuthology.orchestra.run.smithi012.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-16T04:45:24.187 INFO:teuthology.orchestra.run.smithi012.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-16T04:45:24.232 INFO:teuthology.orchestra.run.smithi012.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-16T04:45:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-326-g8921c8ef-1jammy [14.2 kB] 2024-09-16T04:45:24.239 INFO:teuthology.orchestra.run.smithi012.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-326-g8921c8ef-1jammy [51.4 MB] 2024-09-16T04:45:24.269 INFO:teuthology.orchestra.run.smithi012.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-16T04:45:24.270 INFO:teuthology.orchestra.run.smithi012.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-16T04:45:24.274 INFO:teuthology.orchestra.run.smithi012.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-16T04:45:24.276 INFO:teuthology.orchestra.run.smithi012.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-16T04:45:24.293 INFO:teuthology.orchestra.run.smithi012.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-16T04:45:24.294 INFO:teuthology.orchestra.run.smithi012.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-16T04:45:24.313 INFO:teuthology.orchestra.run.smithi012.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-16T04:45:24.318 INFO:teuthology.orchestra.run.smithi012.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-16T04:45:24.320 INFO:teuthology.orchestra.run.smithi012.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-16T04:45:25.352 INFO:teuthology.orchestra.run.smithi012.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-326-g8921c8ef-1jammy [130 kB] 2024-09-16T04:45:25.355 INFO:teuthology.orchestra.run.smithi012.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-326-g8921c8ef-1jammy [39.0 kB] 2024-09-16T04:45:25.356 INFO:teuthology.orchestra.run.smithi012.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-326-g8921c8ef-1jammy [13.6 MB] 2024-09-16T04:45:25.547 INFO:teuthology.orchestra.run.smithi012.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-326-g8921c8ef-1jammy [91.5 kB] 2024-09-16T04:45:26.146 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 181 MB in 4s (50.9 MB/s) 2024-09-16T04:45:26.279 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-16T04:45:26.381 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-16T04:45:26.387 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-16T04:45:26.477 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T04:45:26.772 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-16T04:45:26.799 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-16T04:45:26.839 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T04:45:27.167 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-16T04:45:27.194 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-16T04:45:27.233 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T04:45:27.612 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-16T04:45:27.639 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T04:45:27.678 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T04:45:28.190 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-16T04:45:28.217 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T04:45:28.256 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T04:45:28.585 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-16T04:45:28.612 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T04:45:28.651 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T04:45:28.954 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-16T04:45:28.970 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-16T04:45:29.004 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T04:45:29.290 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librados2. 2024-09-16T04:45:29.317 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../007-librados2_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:29.357 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librados2 (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:29.927 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librbd1. 2024-09-16T04:45:29.953 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../008-librbd1_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:29.993 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librbd1 (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:30.429 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs2. 2024-09-16T04:45:30.456 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:30.496 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs2 (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:30.723 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rados. 2024-09-16T04:45:30.739 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../010-python3-rados_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:30.773 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rados (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:31.093 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-16T04:45:31.120 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-326-g8921c8ef-1jammy_all.deb ... 2024-09-16T04:45:31.159 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-argparse (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:31.411 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cephfs. 2024-09-16T04:45:31.428 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:31.470 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cephfs (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:31.781 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-16T04:45:31.808 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-326-g8921c8ef-1jammy_all.deb ... 2024-09-16T04:45:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-common (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:32.151 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-16T04:45:32.177 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-16T04:45:32.217 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T04:45:32.487 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-prettytable. 2024-09-16T04:45:32.513 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-16T04:45:32.545 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-16T04:45:32.797 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rbd. 2024-09-16T04:45:32.813 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:32.872 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rbd (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:33.242 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-16T04:45:33.269 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-16T04:45:33.309 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T04:45:33.578 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-16T04:45:33.594 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-16T04:45:33.627 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-16T04:45:33.872 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-16T04:45:33.888 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-16T04:45:33.922 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-16T04:45:34.234 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua5.1. 2024-09-16T04:45:34.261 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-16T04:45:34.300 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-16T04:45:34.957 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua-any. 2024-09-16T04:45:34.984 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-16T04:45:35.028 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-16T04:45:35.264 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package zip. 2024-09-16T04:45:35.280 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-16T04:45:35.314 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking zip (3.0-12build2) ... 2024-09-16T04:45:35.617 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package unzip. 2024-09-16T04:45:35.644 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-16T04:45:35.684 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-16T04:45:36.071 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package luarocks. 2024-09-16T04:45:36.098 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-16T04:45:36.137 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-16T04:45:36.537 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librgw2. 2024-09-16T04:45:36.564 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../025-librgw2_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:36.603 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librgw2 (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:37.123 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rgw. 2024-09-16T04:45:37.149 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:37.181 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rgw (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:37.525 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-16T04:45:37.552 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-16T04:45:37.592 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T04:45:37.903 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libradosstriper1. 2024-09-16T04:45:37.930 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:37.970 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libradosstriper1 (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:38.340 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-common. 2024-09-16T04:45:38.367 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../029-ceph-common_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-common (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:39.979 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-base. 2024-09-16T04:45:40.005 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../030-ceph-base_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:40.055 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-base (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:40.607 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-16T04:45:40.636 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-16T04:45:40.674 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-16T04:45:40.902 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cheroot. 2024-09-16T04:45:40.929 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-16T04:45:40.968 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T04:45:41.255 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-16T04:45:41.282 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-16T04:45:41.321 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-16T04:45:41.607 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-16T04:45:41.634 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-16T04:45:41.674 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-16T04:45:41.952 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-16T04:45:41.979 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-16T04:45:42.018 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-16T04:45:42.254 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-tempora. 2024-09-16T04:45:42.270 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-16T04:45:42.304 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-16T04:45:42.607 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-portend. 2024-09-16T04:45:42.634 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-16T04:45:42.674 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-16T04:45:42.996 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-16T04:45:43.020 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-16T04:45:43.060 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-16T04:45:43.354 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-16T04:45:43.381 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-16T04:45:43.412 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-16T04:45:43.758 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-natsort. 2024-09-16T04:45:43.785 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-16T04:45:43.824 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-16T04:45:44.119 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-logutils. 2024-09-16T04:45:44.146 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-16T04:45:44.186 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-16T04:45:44.430 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-16T04:45:44.446 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-16T04:45:44.480 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-16T04:45:44.775 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-mako. 2024-09-16T04:45:44.801 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-16T04:45:44.841 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T04:45:45.320 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-16T04:45:45.346 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-16T04:45:45.386 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-16T04:45:45.689 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-16T04:45:45.716 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-16T04:45:45.755 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-16T04:45:46.017 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webob. 2024-09-16T04:45:46.044 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-16T04:45:46.075 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T04:45:46.353 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-waitress. 2024-09-16T04:45:46.380 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-16T04:45:46.456 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T04:45:46.756 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-tempita. 2024-09-16T04:45:46.782 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-16T04:45:46.822 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T04:45:47.125 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-paste. 2024-09-16T04:45:47.152 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-16T04:45:47.191 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T04:45:47.503 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-16T04:45:47.519 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-16T04:45:47.561 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T04:45:47.839 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-16T04:45:47.866 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-16T04:45:47.897 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-16T04:45:48.176 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webtest. 2024-09-16T04:45:48.202 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-16T04:45:48.242 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-16T04:45:48.553 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pecan. 2024-09-16T04:45:48.580 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-16T04:45:48.619 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T04:45:48.975 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-certifi. 2024-09-16T04:45:49.001 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-16T04:45:49.041 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-16T04:45:49.369 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-idna. 2024-09-16T04:45:49.396 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-16T04:45:49.435 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T04:45:49.713 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-urllib3. 2024-09-16T04:45:49.729 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-16T04:45:49.780 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T04:45:50.075 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-requests. 2024-09-16T04:45:50.101 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-16T04:45:50.133 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T04:45:50.428 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-16T04:45:50.455 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-16T04:45:50.486 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T04:45:50.780 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-16T04:45:50.796 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-326-g8921c8ef-1jammy_all.deb ... 2024-09-16T04:45:50.839 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-modules-core (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:51.219 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-16T04:45:51.244 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:51.285 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:51.596 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-16T04:45:51.614 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-16T04:45:51.654 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-16T04:45:51.932 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-openssl. 2024-09-16T04:45:51.959 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-16T04:45:51.999 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-16T04:45:52.310 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr. 2024-09-16T04:45:52.337 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:52.377 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:52.772 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mon. 2024-09-16T04:45:52.799 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:52.838 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mon (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:53.475 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-osd. 2024-09-16T04:45:53.502 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:53.541 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-osd (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:54.594 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph. 2024-09-16T04:45:54.610 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../066-ceph_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:54.653 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:54.980 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-fuse. 2024-09-16T04:45:55.008 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:55.047 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-fuse (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:55.967 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mds. 2024-09-16T04:45:55.995 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:56.317 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mds (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:56.753 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephadm. 2024-09-16T04:45:56.781 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../069-cephadm_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:45:56.820 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephadm (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:57.131 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-16T04:45:57.158 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-16T04:45:57.197 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-16T04:45:57.502 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-16T04:45:57.518 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T04:45:57.560 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T04:45:58.490 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-babel. 2024-09-16T04:45:58.503 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T04:45:58.531 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T04:45:58.810 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jinja2. 2024-09-16T04:45:58.837 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-16T04:45:58.876 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T04:45:59.179 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-16T04:45:59.206 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-326-g8921c8ef-1jammy_all.deb ... 2024-09-16T04:45:59.245 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-cephadm (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:45:59.532 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-16T04:45:59.548 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-16T04:45:59.582 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-16T04:45:59.920 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-routes. 2024-09-16T04:45:59.946 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-16T04:45:59.985 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T04:46:00.296 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-16T04:46:00.324 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-326-g8921c8ef-1jammy_all.deb ... 2024-09-16T04:46:00.363 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-dashboard (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:01.585 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-16T04:46:01.601 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-16T04:46:01.643 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T04:46:02.130 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-joblib. 2024-09-16T04:46:02.157 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-16T04:46:02.196 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T04:46:02.558 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-16T04:46:02.574 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-16T04:46:02.607 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-16T04:46:02.885 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn. 2024-09-16T04:46:02.913 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-16T04:46:02.952 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T04:46:03.719 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-16T04:46:03.747 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-326-g8921c8ef-1jammy_all.deb ... 2024-09-16T04:46:03.872 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:04.799 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cachetools. 2024-09-16T04:46:04.827 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-16T04:46:04.857 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-16T04:46:05.127 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-16T04:46:05.144 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-16T04:46:05.185 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-16T04:46:05.513 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-16T04:46:05.541 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-16T04:46:05.580 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T04:46:06.066 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rsa. 2024-09-16T04:46:06.094 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-16T04:46:06.132 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-16T04:46:06.478 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-google-auth. 2024-09-16T04:46:06.506 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-16T04:46:06.544 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-16T04:46:06.864 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-16T04:46:06.893 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-16T04:46:06.931 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T04:46:07.242 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-websocket. 2024-09-16T04:46:07.270 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-16T04:46:07.308 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-16T04:46:07.629 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-16T04:46:07.657 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-16T04:46:07.695 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T04:46:08.384 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-16T04:46:08.412 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-326-g8921c8ef-1jammy_all.deb ... 2024-09-16T04:46:08.459 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:08.736 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-16T04:46:08.753 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-16T04:46:08.786 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T04:46:09.123 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-16T04:46:09.150 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T04:46:09.189 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T04:46:09.450 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package jq. 2024-09-16T04:46:09.475 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T04:46:09.516 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-16T04:46:09.778 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package socat. 2024-09-16T04:46:09.806 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-16T04:46:09.836 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-16T04:46:10.131 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package xmlstarlet. 2024-09-16T04:46:10.147 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-16T04:46:10.189 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-16T04:46:10.552 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-test. 2024-09-16T04:46:10.580 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../097-ceph-test_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:46:10.618 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-test (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:12.939 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-volume. 2024-09-16T04:46:12.955 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-326-g8921c8ef-1jammy_all.deb ... 2024-09-16T04:46:12.989 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-volume (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:13.284 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-16T04:46:13.301 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:46:13.334 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs-dev (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:13.587 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-16T04:46:13.615 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-16T04:46:13.653 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-16T04:46:13.958 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-16T04:46:13.986 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-16T04:46:14.024 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-16T04:46:14.345 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package nvme-cli. 2024-09-16T04:46:14.373 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-16T04:46:14.403 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T04:46:14.750 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package pkg-config. 2024-09-16T04:46:14.778 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-16T04:46:14.808 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-16T04:46:15.069 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-16T04:46:15.086 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-16T04:46:15.127 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-16T04:46:15.531 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-16T04:46:15.708 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-16T04:46:15.708 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-16T04:46:16.058 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastescript. 2024-09-16T04:46:16.085 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-16T04:46:16.124 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-16T04:46:16.470 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pluggy. 2024-09-16T04:46:16.498 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-16T04:46:16.536 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-16T04:46:16.831 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-psutil. 2024-09-16T04:46:16.859 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-16T04:46:16.897 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-16T04:46:17.242 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-py. 2024-09-16T04:46:17.270 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-16T04:46:17.316 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-16T04:46:17.612 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pygments. 2024-09-16T04:46:17.640 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-16T04:46:17.670 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T04:46:18.099 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-16T04:46:18.127 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-16T04:46:18.165 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-16T04:46:18.477 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-toml. 2024-09-16T04:46:18.505 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-16T04:46:18.543 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-16T04:46:18.846 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pytest. 2024-09-16T04:46:18.874 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-16T04:46:18.912 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T04:46:19.258 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplejson. 2024-09-16T04:46:19.286 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-16T04:46:19.324 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-16T04:46:19.585 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-16T04:46:19.602 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-16T04:46:19.635 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-16T04:46:20.372 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package radosgw. 2024-09-16T04:46:20.401 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../116-radosgw_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:46:20.439 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking radosgw (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:21.520 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package rbd-fuse. 2024-09-16T04:46:21.548 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-326-g8921c8ef-1jammy_amd64.deb ... 2024-09-16T04:46:21.591 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking rbd-fuse (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:21.888 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-16T04:46:22.155 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T04:46:22.273 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T04:46:22.512 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-16T04:46:22.847 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-16T04:46:23.288 INFO:teuthology.orchestra.run.smithi012.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-16T04:46:23.349 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephadm (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:23.874 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user cephadm....done 2024-09-16T04:46:24.013 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T04:46:24.331 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-16T04:46:24.566 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-16T04:46:24.701 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-16T04:46:24.952 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-16T04:46:25.219 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T04:46:25.345 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-16T04:46:25.664 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T04:46:26.032 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-16T04:46:26.292 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-16T04:46:26.629 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-16T04:46:27.019 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-16T04:46:27.279 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-16T04:46:27.538 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-16T04:46:27.822 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-argparse (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:28.073 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T04:46:28.208 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-16T04:46:28.334 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-16T04:46:28.460 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-16T04:46:28.719 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-16T04:46:29.098 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-16T04:46:29.134 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-16T04:46:29.198 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T04:46:29.301 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-16T04:46:29.650 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T04:46:29.793 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-16T04:46:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T04:46:30.336 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-16T04:46:30.646 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up zip (3.0-12build2) ... 2024-09-16T04:46:30.772 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T04:46:31.482 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T04:46:31.750 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T04:46:31.876 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-16T04:46:32.002 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T04:46:32.303 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-16T04:46:32.713 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-16T04:46:32.955 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T04:46:33.349 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T04:46:33.625 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T04:46:33.993 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-16T04:46:34.253 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-16T04:46:34.378 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-common (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:34.663 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T04:46:34.972 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T04:46:36.228 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-16T04:46:36.505 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T04:46:36.622 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-16T04:46:37.065 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T04:46:37.183 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-16T04:46:37.284 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T04:46:37.569 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-16T04:46:37.828 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-16T04:46:38.079 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T04:46:38.197 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-16T04:46:38.506 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-16T04:46:38.766 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-16T04:46:39.026 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-16T04:46:39.277 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-16T04:46:39.528 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-16T04:46:39.787 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T04:46:39.896 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-16T04:46:40.189 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T04:46:40.316 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T04:46:40.609 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T04:46:40.909 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-16T04:46:41.169 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-16T04:46:41.287 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-16T04:46:41.391 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T04:46:41.486 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-16T04:46:41.731 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T04:46:41.857 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T04:46:42.242 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-16T04:46:42.510 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-16T04:46:42.636 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-16T04:46:42.895 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T04:46:43.046 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T04:46:43.364 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T04:46:43.665 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T04:46:43.941 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-16T04:46:44.050 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-16T04:46:44.327 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-16T04:46:44.603 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T04:46:44.931 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-16T04:46:45.327 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-16T04:46:45.603 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T04:46:45.947 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T04:46:46.073 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librados2 (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:46.199 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libsqlite3-mod-ceph (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:46.359 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-16T04:46:46.679 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs2 (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:46.779 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libradosstriper1 (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:46.905 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-16T04:46:47.190 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librbd1 (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:47.291 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-modules-core (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:47.409 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-fuse (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:47.629 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T04:46:47.629 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T04:46:48.077 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs-dev (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:48.187 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rados (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:48.296 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librgw2 (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:48.422 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T04:46:49.684 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rbd (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:49.810 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up rbd-fuse (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:49.920 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rgw (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:50.021 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cephfs (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:50.147 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-common (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:50.674 INFO:teuthology.orchestra.run.smithi012.stdout:Adding group ceph....done 2024-09-16T04:46:51.025 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user ceph....done 2024-09-16T04:46:51.194 INFO:teuthology.orchestra.run.smithi012.stdout:Setting system user ceph properties....done 2024-09-16T04:46:51.210 INFO:teuthology.orchestra.run.smithi012.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-16T04:46:51.350 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-16T04:46:51.676 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-16T04:46:52.140 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-test (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:52.291 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up radosgw (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:52.779 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T04:46:52.780 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T04:46:53.268 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-base (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:53.606 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-16T04:46:54.070 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mds (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:54.305 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T04:46:54.305 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T04:46:54.797 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:55.023 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T04:46:55.023 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T04:46:55.515 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-osd (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:55.813 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T04:46:55.813 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T04:46:56.315 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-k8sevents (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:56.443 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:56.577 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mon (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:56.780 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T04:46:56.780 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T04:46:57.295 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-cephadm (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:57.489 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:57.615 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-dashboard (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:57.774 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-volume (19.1.1-326-g8921c8ef-1jammy) ... 2024-09-16T04:46:58.236 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-16T04:46:58.345 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-16T04:46:58.497 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-16T04:46:59.573 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-16T04:47:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-16T04:47:00.237 DEBUG:teuthology.parallel:result is None 2024-09-16T04:47:00.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T04:47:00.385 DEBUG:teuthology.orchestra.run.smithi012:> dpkg-query -W -f '${Version}' ceph 2024-09-16T04:47:00.405 INFO:teuthology.orchestra.run.smithi012.stdout:19.1.1-326-g8921c8ef-1jammy 2024-09-16T04:47:00.405 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326-g8921c8ef-1jammy 2024-09-16T04:47:00.405 INFO:teuthology.task.install:The correct ceph version 19.1.1-326-g8921c8ef-1jammy is installed. 2024-09-16T04:47:00.407 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T04:47:00.407 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:00.407 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T04:47:00.463 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T04:47:00.463 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:00.463 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T04:47:00.518 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T04:47:00.573 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T04:47:00.573 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:00.573 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T04:47:00.629 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T04:47:00.685 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T04:47:00.685 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:00.685 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T04:47:00.741 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T04:47:00.797 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T04:47:00.893 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T04:47:00.894 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /var/log/ceph 2024-09-16T04:47:00.908 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T04:47:00.908 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T04:47:00.965 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T04:47:00.966 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T04:47:01.031 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T04:47:01.032 INFO:tasks.ceph:config {'conf': {'global': {'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2024-09-16T04:47:01.032 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_21:00:02-rados-squid-distro-default-smithi/7906051', 'branch': 'squid', 'description': 'rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7906051', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-15_21:00:02-rados-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'mon_bind_msgr2': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ad35db06e8924912f01e32b35576297c0f1352b4'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 700, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2']], 'seed': 5616, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'sleep_before_teardown': 0, 'subset': '15927/100000', 'suite': 'rados', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ad35db06e8924912f01e32b35576297c0f1352b4', 'targets': {'smithi012.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCJW9FBA3NDU5n9ipFsaafiiQByl+wKwFnjlgvHWvjSfnEH6m9uAJTiriqemXBzceYOjBn5okI12ySN4wGNeW+A='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'conf': {'global': {'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'mon_bind_msgr2': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'cluster': 'ceph'}}, {'rep_lost_unfound_delete': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_21:00:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T04:47:01.033 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T04:47:01.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T04:47:01.128 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:01.128 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T04:47:01.178 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T04:47:01.178 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-09-16T04:47:01.226 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T04:47:01.226 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T04:47:01.226 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-09-16T04:47:01.226 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T04:47:01.226 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-16 04:41:35.529120353 +0000 2024-09-16T04:47:01.226 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-16 04:41:35.265122711 +0000 2024-09-16T04:47:01.226 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-16 04:41:35.265122711 +0000 2024-09-16T04:47:01.227 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-09-16T04:47:01.227 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T04:47:01.280 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-16T04:47:01.281 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-16T04:47:01.281 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000443286 s, 1.2 MB/s 2024-09-16T04:47:01.282 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T04:47:01.331 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-09-16T04:47:01.377 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T04:47:01.378 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T04:47:01.378 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 900 Links: 1 2024-09-16T04:47:01.378 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T04:47:01.378 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-16 04:41:36.041115779 +0000 2024-09-16T04:47:01.378 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-16 04:41:35.765118244 +0000 2024-09-16T04:47:01.378 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-16 04:41:35.765118244 +0000 2024-09-16T04:47:01.378 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-09-16T04:47:01.378 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T04:47:01.432 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-16T04:47:01.432 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-16T04:47:01.432 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000364042 s, 1.4 MB/s 2024-09-16T04:47:01.434 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T04:47:01.487 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-09-16T04:47:01.538 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T04:47:01.538 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T04:47:01.539 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-09-16T04:47:01.539 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T04:47:01.539 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-16 04:41:36.537111349 +0000 2024-09-16T04:47:01.539 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-16 04:41:36.285113600 +0000 2024-09-16T04:47:01.539 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-16 04:41:36.285113600 +0000 2024-09-16T04:47:01.540 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-09-16T04:47:01.540 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T04:47:01.592 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-16T04:47:01.593 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-16T04:47:01.593 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000379778 s, 1.3 MB/s 2024-09-16T04:47:01.594 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T04:47:01.643 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-09-16T04:47:01.689 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T04:47:01.690 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T04:47:01.690 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-09-16T04:47:01.690 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T04:47:01.690 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-16 04:41:37.053106741 +0000 2024-09-16T04:47:01.690 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-16 04:41:36.801108990 +0000 2024-09-16T04:47:01.690 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-16 04:41:36.801108990 +0000 2024-09-16T04:47:01.690 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-09-16T04:47:01.690 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T04:47:01.744 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-16T04:47:01.744 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-16T04:47:01.744 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000324815 s, 1.6 MB/s 2024-09-16T04:47:01.745 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T04:47:01.795 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T04:47:01.795 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-16T04:47:01.795 INFO:tasks.ceph:Generating config... 2024-09-16T04:47:01.797 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-09-16T04:47:01.797 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-09-16T04:47:01.797 INFO:tasks.ceph:[global] ms type = async 2024-09-16T04:47:01.797 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T04:47:01.797 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T04:47:01.797 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T04:47:01.797 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T04:47:01.797 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T04:47:01.797 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-16T04:47:01.797 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-16T04:47:01.797 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-16T04:47:01.798 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T04:47:01.799 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-16T04:47:01.799 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-16T04:47:01.799 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-09-16T04:47:01.799 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T04:47:01.799 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-16T04:47:01.799 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-16T04:47:01.799 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T04:47:01.799 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T04:47:01.875 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T04:47:01.883 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T04:47:01.967 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T04:47:02.025 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.12:6789]'), ('mon.b', '[v1:172.21.15.12:6790]'), ('mon.c', '[v1:172.21.15.12:6791]')] 2024-09-16T04:47:02.026 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.12:6789],[v1:172.21.15.12:6790],[v1:172.21.15.12:6791]', 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-16T04:47:02.027 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:02.027 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T04:47:02.077 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.12:6789]' --addv b '[v1:172.21.15.12:6790]' --addv c '[v1:172.21.15.12:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T04:47:02.166 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T04:47:02.166 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: generated fsid b9a92eba-c518-4afa-ae65-ed2825dd0c06 2024-09-16T04:47:02.166 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = quincy 2024-09-16T04:47:02.166 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 0 2024-09-16T04:47:02.167 INFO:teuthology.orchestra.run.smithi012.stdout:fsid b9a92eba-c518-4afa-ae65-ed2825dd0c06 2024-09-16T04:47:02.167 INFO:teuthology.orchestra.run.smithi012.stdout:last_changed 2024-09-16T04:47:02.165652+0000 2024-09-16T04:47:02.167 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-09-16T04:47:02.165652+0000 2024-09-16T04:47:02.167 INFO:teuthology.orchestra.run.smithi012.stdout:min_mon_release 17 (quincy) 2024-09-16T04:47:02.167 INFO:teuthology.orchestra.run.smithi012.stdout:election_strategy: 3 2024-09-16T04:47:02.167 INFO:teuthology.orchestra.run.smithi012.stdout:0: v1:172.21.15.12:6789/0 mon.a 2024-09-16T04:47:02.167 INFO:teuthology.orchestra.run.smithi012.stdout:1: v1:172.21.15.12:6790/0 mon.b 2024-09-16T04:47:02.167 INFO:teuthology.orchestra.run.smithi012.stdout:2: v1:172.21.15.12:6791/0 mon.c 2024-09-16T04:47:02.167 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T04:47:02.169 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T04:47:02.174 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b9a92eba-c518-4afa-ae65-ed2825dd0c06... 2024-09-16T04:47:02.175 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T04:47:02.245 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-09-16T04:47:02.245 INFO:teuthology.orchestra.run.smithi012.stdout: chdir = "" 2024-09-16T04:47:02.245 INFO:teuthology.orchestra.run.smithi012.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T04:47:02.245 INFO:teuthology.orchestra.run.smithi012.stdout: auth supported = cephx 2024-09-16T04:47:02.245 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.245 INFO:teuthology.orchestra.run.smithi012.stdout: filestore xattr use omap = true 2024-09-16T04:47:02.245 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.245 INFO:teuthology.orchestra.run.smithi012.stdout: mon clock drift allowed = 1.000 2024-09-16T04:47:02.245 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.245 INFO:teuthology.orchestra.run.smithi012.stdout: osd crush chooseleaf type = 0 2024-09-16T04:47:02.246 INFO:teuthology.orchestra.run.smithi012.stdout: auth debug = true 2024-09-16T04:47:02.246 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.246 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on old message = true 2024-09-16T04:47:02.246 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on bug = true 2024-09-16T04:47:02.246 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.246 INFO:teuthology.orchestra.run.smithi012.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T04:47:02.246 INFO:teuthology.orchestra.run.smithi012.stdout: mon pg warn max object skew = 0 2024-09-16T04:47:02.246 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.246 INFO:teuthology.orchestra.run.smithi012.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T04:47:02.246 INFO:teuthology.orchestra.run.smithi012.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T04:47:02.246 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.246 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default size = 2 2024-09-16T04:47:02.247 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.247 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow primary affinity = true 2024-09-16T04:47:02.247 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow pg remap = true 2024-09-16T04:47:02.247 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on legacy crush tunables = false 2024-09-16T04:47:02.247 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on crush straw calc version zero = false 2024-09-16T04:47:02.247 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on no sortbitwise = false 2024-09-16T04:47:02.247 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on osd down out interval zero = false 2024-09-16T04:47:02.247 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on too few osds = false 2024-09-16T04:47:02.247 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T04:47:02.247 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T04:47:02.247 INFO:teuthology.orchestra.run.smithi012.stdout: mon_allow_pool_size_one = true 2024-09-16T04:47:02.247 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.247 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T04:47:02.248 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.248 INFO:teuthology.orchestra.run.smithi012.stdout: osd default data pool replay window = 5 2024-09-16T04:47:02.248 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.248 INFO:teuthology.orchestra.run.smithi012.stdout: mon allow pool delete = true 2024-09-16T04:47:02.248 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.248 INFO:teuthology.orchestra.run.smithi012.stdout: mon cluster log file level = debug 2024-09-16T04:47:02.248 INFO:teuthology.orchestra.run.smithi012.stdout: debug asserts on shutdown = true 2024-09-16T04:47:02.248 INFO:teuthology.orchestra.run.smithi012.stdout: mon health detail to clog = false 2024-09-16T04:47:02.248 INFO:teuthology.orchestra.run.smithi012.stdout: mon host = "[v1:172.21.15.12:6789],[v1:172.21.15.12:6790],[v1:172.21.15.12:6791]" 2024-09-16T04:47:02.248 INFO:teuthology.orchestra.run.smithi012.stdout: mon election default strategy = 3 2024-09-16T04:47:02.248 INFO:teuthology.orchestra.run.smithi012.stdout: ms bind msgr2 = False 2024-09-16T04:47:02.248 INFO:teuthology.orchestra.run.smithi012.stdout: ms type = async 2024-09-16T04:47:02.249 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = b9a92eba-c518-4afa-ae65-ed2825dd0c06 2024-09-16T04:47:02.249 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.249 INFO:teuthology.orchestra.run.smithi012.stdout:[osd] 2024-09-16T04:47:02.249 INFO:teuthology.orchestra.run.smithi012.stdout: osd journal size = 100 2024-09-16T04:47:02.249 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.249 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub load threshold = 5.0 2024-09-16T04:47:02.249 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub max interval = 600 2024-09-16T04:47:02.249 INFO:teuthology.orchestra.run.smithi012.stdout: osd mclock profile = high_recovery_ops 2024-09-16T04:47:02.249 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.249 INFO:teuthology.orchestra.run.smithi012.stdout: osd recover clone overlap = true 2024-09-16T04:47:02.249 INFO:teuthology.orchestra.run.smithi012.stdout: osd recovery max chunk = 1048576 2024-09-16T04:47:02.249 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug shutdown = true 2024-09-16T04:47:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug op order = true 2024-09-16T04:47:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug verify stray on activate = true 2024-09-16T04:47:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug trim objects = true 2024-09-16T04:47:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: osd open classes on start = true 2024-09-16T04:47:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug pg log writeout = true 2024-09-16T04:47:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: osd deep scrub update digest min age = 30 2024-09-16T04:47:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: osd map max advance = 10 2024-09-16T04:47:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.250 INFO:teuthology.orchestra.run.smithi012.stdout: journal zero on create = true 2024-09-16T04:47:02.251 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.251 INFO:teuthology.orchestra.run.smithi012.stdout: filestore ondisk finisher threads = 3 2024-09-16T04:47:02.251 INFO:teuthology.orchestra.run.smithi012.stdout: filestore apply finisher threads = 3 2024-09-16T04:47:02.251 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.251 INFO:teuthology.orchestra.run.smithi012.stdout: bdev debug aio = true 2024-09-16T04:47:02.251 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug misdirected ops = true 2024-09-16T04:47:02.251 INFO:teuthology.orchestra.run.smithi012.stdout: bdev async discard = True 2024-09-16T04:47:02.251 INFO:teuthology.orchestra.run.smithi012.stdout: bdev enable discard = True 2024-09-16T04:47:02.251 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore block size = 96636764160 2024-09-16T04:47:02.251 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore fsck on mount = True 2024-09-16T04:47:02.251 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore zero block detection = True 2024-09-16T04:47:02.251 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluefs = 20 2024-09-16T04:47:02.251 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluestore = 20 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: debug osd = 20 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: debug rocksdb = 10 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd full ratio = 0.9 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd nearfull ratio = 0.8 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug verify cached snaps = True 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug verify missing on start = True 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: osd failsafe full ratio = 0.95 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: osd mclock override recovery settings = True 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: osd memory target = 1610612736 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: osd objectstore = bluestore 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: osd op queue = debug_random 2024-09-16T04:47:02.252 INFO:teuthology.orchestra.run.smithi012.stdout: osd op queue cut off = debug_random 2024-09-16T04:47:02.253 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.253 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr] 2024-09-16T04:47:02.253 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-09-16T04:47:02.253 INFO:teuthology.orchestra.run.smithi012.stdout: debug mgr = 20 2024-09-16T04:47:02.253 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2024-09-16T04:47:02.253 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2024-09-16T04:47:02.253 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2024-09-16T04:47:02.253 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2024-09-16T04:47:02.253 INFO:teuthology.orchestra.run.smithi012.stdout: mgr/telemetry/nag = false 2024-09-16T04:47:02.253 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.253 INFO:teuthology.orchestra.run.smithi012.stdout:[mon] 2024-09-16T04:47:02.253 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-09-16T04:47:02.253 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: debug paxos = 20 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: mon data avail warn = 5 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: mon mgr mkfs grace = 240 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd reporter subtree level = osd 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd prime pg temp = true 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.254 INFO:teuthology.orchestra.run.smithi012.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T04:47:02.255 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T04:47:02.255 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T04:47:02.255 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.255 INFO:teuthology.orchestra.run.smithi012.stdout: # 1m isn't quite enough 2024-09-16T04:47:02.255 INFO:teuthology.orchestra.run.smithi012.stdout: mon_down_mkfs_grace = 2m 2024-09-16T04:47:02.255 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.255 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_filestore_osds = false 2024-09-16T04:47:02.255 INFO:teuthology.orchestra.run.smithi012.stdout: mon scrub interval = 300 2024-09-16T04:47:02.255 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:02.255 INFO:teuthology.orchestra.run.smithi012.stdout:[client] 2024-09-16T04:47:02.255 INFO:teuthology.orchestra.run.smithi012.stdout: rgw cache enabled = true 2024-09-16T04:47:02.255 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable ops log = true 2024-09-16T04:47:02.256 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable usage log = true 2024-09-16T04:47:02.256 INFO:teuthology.orchestra.run.smithi012.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T04:47:02.256 INFO:teuthology.orchestra.run.smithi012.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T04:47:02.256 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.a] 2024-09-16T04:47:02.256 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.b] 2024-09-16T04:47:02.256 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.c] 2024-09-16T04:47:02.257 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T04:47:02.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T04:47:02.355 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T04:47:02.355 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:02.355 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T04:47:02.360 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:02.360 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T04:47:02.405 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi012.front.sepia.ceph.com 2024-09-16T04:47:02.406 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:02.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T04:47:02.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T04:47:02.473 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:02.473 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T04:47:02.522 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T04:47:02.522 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T04:47:02.522 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T04:47:02.608 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T04:47:02.611 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T04:47:02.612 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T04:47:02.612 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T04:47:02.612 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-16T04:47:02.612 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T04:47:02.664 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T04:47:02.665 INFO:tasks.ceph:role: osd.0 2024-09-16T04:47:02.665 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-16T04:47:02.665 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T04:47:02.730 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T04:47:02.731 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T04:47:02.731 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T04:47:02.731 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T04:47:02.731 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T04:47:02.731 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-09-16T04:47:02.731 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T04:47:02.731 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T04:47:02.731 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T04:47:02.731 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T04:47:03.211 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2024-09-16T04:47:03.213 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-09-16T04:47:03.214 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T04:47:03.365 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T04:47:03.376 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T04:47:03.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:47:03.377 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T04:47:03.433 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T04:47:03.433 INFO:tasks.ceph:role: osd.1 2024-09-16T04:47:03.434 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-16T04:47:03.434 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T04:47:03.497 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T04:47:03.498 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T04:47:03.498 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T04:47:03.498 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T04:47:03.498 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T04:47:03.498 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-09-16T04:47:03.498 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T04:47:03.498 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T04:47:03.498 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T04:47:03.498 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T04:47:03.937 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2024-09-16T04:47:03.939 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-09-16T04:47:03.939 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T04:47:04.022 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T04:47:04.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:47:04.077 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T04:47:04.077 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T04:47:04.133 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T04:47:04.133 INFO:tasks.ceph:role: osd.2 2024-09-16T04:47:04.133 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-16T04:47:04.133 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T04:47:04.197 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T04:47:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T04:47:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T04:47:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T04:47:04.198 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T04:47:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-09-16T04:47:04.198 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T04:47:04.198 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T04:47:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T04:47:04.198 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T04:47:04.662 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2024-09-16T04:47:04.664 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-09-16T04:47:04.664 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T04:47:04.793 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T04:47:04.805 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T04:47:04.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:47:04.806 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T04:47:04.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-16T04:47:04.918+0000 7f2e109697c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T04:47:04.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-16T04:47:04.922+0000 7f2e109697c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T04:47:04.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-16T04:47:04.922+0000 7f2e109697c0 -1 bdev(0x556311bb7800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-16T04:47:04.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-16T04:47:04.922+0000 7f2e109697c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T04:47:09.054 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T04:47:09.069 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T04:47:09.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-16T04:47:09.146+0000 7f4ba144d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T04:47:09.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-16T04:47:09.146+0000 7f4ba144d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T04:47:09.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-16T04:47:09.150+0000 7f4ba144d7c0 -1 bdev(0x55cdd123b800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-16T04:47:09.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-16T04:47:09.150+0000 7f4ba144d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T04:47:12.281 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T04:47:12.338 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T04:47:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-16T04:47:12.418+0000 7fe5388087c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T04:47:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-16T04:47:12.418+0000 7fe5388087c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T04:47:12.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-16T04:47:12.418+0000 7fe5388087c0 -1 bdev(0x558c2f879800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-16T04:47:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-16T04:47:12.418+0000 7fe5388087c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T04:47:15.811 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T04:47:15.867 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T04:47:15.867 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:15.867 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T04:47:15.922 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:15.922 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T04:47:15.978 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:15.978 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T04:47:16.035 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:16.035 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T04:47:16.089 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T04:47:16.089 DEBUG:teuthology.orchestra.run.smithi012:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T04:47:16.143 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr.x] 2024-09-16T04:47:16.143 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQBGuOdmvuBBJBAAlzFRkPIWxnlAsbEi3jm63Q== 2024-09-16T04:47:16.143 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.0] 2024-09-16T04:47:16.143 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQBIuOdmwAsJNxAALiwuuOjWJ0lHuldHUJ8Xpw== 2024-09-16T04:47:16.144 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.1] 2024-09-16T04:47:16.144 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQBNuOdmGCgQCRAAsASYMPRanwPWUE8AMkMFqg== 2024-09-16T04:47:16.144 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.2] 2024-09-16T04:47:16.144 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQBQuOdmvZcfGRAAF7ZgyNeV23jepoe0N/nMTw== 2024-09-16T04:47:16.145 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T04:47:16.236 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T04:47:16.319 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T04:47:16.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T04:47:16.449 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T04:47:16.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T04:47:16.503 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T04:47:16.596 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T04:47:16.606 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T04:47:16.658 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T04:47:16.751 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T04:47:16.761 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T04:47:16.816 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T04:47:16.906 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T04:47:16.917 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T04:47:16.965 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T04:47:16.965 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T04:47:16.965 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T04:47:17.010 INFO:tasks.ceph.mon.a:Started 2024-09-16T04:47:17.010 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T04:47:17.011 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T04:47:17.013 INFO:tasks.ceph.mon.b:Started 2024-09-16T04:47:17.013 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T04:47:17.013 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T04:47:17.015 INFO:tasks.ceph.mon.c:Started 2024-09-16T04:47:17.015 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph config set mgr mgr_pool false --force 2024-09-16T04:47:27.301 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T04:47:27.301 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T04:47:27.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T04:47:27.304 INFO:tasks.ceph.mgr.x:Started 2024-09-16T04:47:27.304 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T04:47:27.304 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T04:47:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-09-16T04:47:27.520 INFO:tasks.ceph:check_enable_crimson: False 2024-09-16T04:47:27.521 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T04:47:27.521 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:27.521 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T04:47:27.534 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:27.534 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T04:47:27.584 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:27.578+0000 7f3e02b4b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T04:47:27.588 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-16T04:47:27.588 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T04:47:27.640 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd new 7fa4b8a1-d218-47cb-adc4-469674c829fc 0 2024-09-16T04:47:27.881 INFO:teuthology.orchestra.run.smithi012.stdout:0 2024-09-16T04:47:27.898 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd new 7234f3ae-2cf3-4dd4-8c21-4f170976dbf4 1 2024-09-16T04:47:28.072 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:28.066+0000 7f3e02b4b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T04:47:28.095 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-09-16T04:47:28.111 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd new 919a351e-ecee-4e06-bff1-ae68c80d320f 2 2024-09-16T04:47:28.276 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:28.270+0000 7f3e02b4b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T04:47:28.318 INFO:teuthology.orchestra.run.smithi012.stdout:2 2024-09-16T04:47:28.334 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T04:47:28.334 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T04:47:28.337 INFO:tasks.ceph.osd.0:Started 2024-09-16T04:47:28.337 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T04:47:28.337 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T04:47:28.339 INFO:tasks.ceph.osd.1:Started 2024-09-16T04:47:28.339 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T04:47:28.339 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T04:47:28.341 INFO:tasks.ceph.osd.2:Started 2024-09-16T04:47:28.342 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T04:47:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:28.519 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":5,"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","created":"2024-09-16T04:47:27.160076+0000","modified":"2024-09-16T04:47:28.313884+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T04:47:28.313884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7fa4b8a1-d218-47cb-adc4-469674c829fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7234f3ae-2cf3-4dd4-8c21-4f170976dbf4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"919a351e-ecee-4e06-bff1-ae68c80d320f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T04:47:28.536 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T04:47:28.537 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T04:47:28.838 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T04:47:28.891 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:28.886+0000 7f3e02b4b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T04:47:28.949 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:28.946+0000 7f3e02b4b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T04:47:29.005 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:29.002+0000 7f3e02b4b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T04:47:29.011 INFO:teuthology.misc.health.smithi012.stdout: 2024-09-16T04:47:29.011 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":5,"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","created":"2024-09-16T04:47:27.160076+0000","modified":"2024-09-16T04:47:28.313884+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T04:47:28.313884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7fa4b8a1-d218-47cb-adc4-469674c829fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7234f3ae-2cf3-4dd4-8c21-4f170976dbf4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"919a351e-ecee-4e06-bff1-ae68c80d320f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T04:47:29.025 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-16T04:47:29.087 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:29.082+0000 7f3e02b4b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T04:47:29.142 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:29.138+0000 7f3e02b4b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T04:47:29.224 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:29.218+0000 7f3e02b4b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T04:47:29.280 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:29.274+0000 7f3e02b4b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T04:47:29.438 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:29.434+0000 7f3e02b4b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T04:47:29.503 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:29.498+0000 7f3e02b4b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T04:47:29.971 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-16T04:47:29.966+0000 7f3d1bf277c0 -1 Falling back to public interface 2024-09-16T04:47:29.998 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:47:29.994+0000 7fc9971767c0 -1 Falling back to public interface 2024-09-16T04:47:30.148 INFO:tasks.ceph.mgr.x.smithi012.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-16T04:47:30.148 INFO:tasks.ceph.mgr.x.smithi012.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-16T04:47:30.148 INFO:tasks.ceph.mgr.x.smithi012.stderr: from numpy import show_config as show_numpy_config 2024-09-16T04:47:30.219 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:30.214+0000 7f3e02b4b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T04:47:30.239 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-16T04:47:30.234+0000 7f9018df37c0 -1 Falling back to public interface 2024-09-16T04:47:30.410 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:30.406+0000 7f3e02b4b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T04:47:30.590 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:30.586+0000 7f3e02b4b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T04:47:30.661 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:30.658+0000 7f3e02b4b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T04:47:30.813 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:30.810+0000 7f3e02b4b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T04:47:31.464 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:31.458+0000 7f3e02b4b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T04:47:31.545 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:31.538+0000 7f3e02b4b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T04:47:31.736 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:31.730+0000 7f3e02b4b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T04:47:31.813 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:31.810+0000 7f3e02b4b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T04:47:32.121 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:32.118+0000 7f3e02b4b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T04:47:32.188 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:32.182+0000 7f3e02b4b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T04:47:33.257 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:33.249+0000 7f3da9780640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T04:47:33.257 INFO:tasks.ceph.mgr.x.smithi012.stderr:2024-09-16T04:47:33.253+0000 7f3da9780640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T04:47:34.945 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-16T04:47:34.941+0000 7f9018df37c0 -1 osd.1 0 log_to_monitors true 2024-09-16T04:47:35.037 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-16T04:47:35.033+0000 7f3d1bf277c0 -1 osd.2 0 log_to_monitors true 2024-09-16T04:47:35.113 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:47:35.109+0000 7fc9971767c0 -1 osd.0 0 log_to_monitors true 2024-09-16T04:47:35.327 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T04:47:35.627 INFO:teuthology.misc.health.smithi012.stdout: 2024-09-16T04:47:35.627 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":6,"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","created":"2024-09-16T04:47:27.160076+0000","modified":"2024-09-16T04:47:35.292000+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T04:47:28.313884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7fa4b8a1-d218-47cb-adc4-469674c829fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7234f3ae-2cf3-4dd4-8c21-4f170976dbf4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"919a351e-ecee-4e06-bff1-ae68c80d320f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T04:47:35.645 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-16T04:47:36.316 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:47:36.309+0000 7fc9928fc640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T04:47:36.381 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:47:36.373+0000 7fc98ef4b640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T04:47:37.674 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-16T04:47:37.669+0000 7f3d186e2640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T04:47:37.679 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-16T04:47:37.673+0000 7f3d0fcf4640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T04:47:37.733 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-16T04:47:37.725+0000 7f90155ae640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T04:47:37.737 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-16T04:47:37.729+0000 7f900cbc0640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T04:47:41.947 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T04:47:42.250 INFO:teuthology.misc.health.smithi012.stdout: 2024-09-16T04:47:42.251 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":10,"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","created":"2024-09-16T04:47:27.160076+0000","modified":"2024-09-16T04:47:39.680698+0000","last_up_change":"2024-09-16T04:47:38.678442+0000","last_in_change":"2024-09-16T04:47:28.313884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7fa4b8a1-d218-47cb-adc4-469674c829fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6804","nonce":1112855623}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6805","nonce":1112855623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6807","nonce":1112855623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6806","nonce":1112855623}]},"public_addr":"172.21.15.12:6804/1112855623","cluster_addr":"172.21.15.12:6805/1112855623","heartbeat_back_addr":"172.21.15.12:6807/1112855623","heartbeat_front_addr":"172.21.15.12:6806/1112855623","state":["exists","up"]},{"osd":1,"uuid":"7234f3ae-2cf3-4dd4-8c21-4f170976dbf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6808","nonce":1033795119}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6809","nonce":1033795119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6811","nonce":1033795119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6810","nonce":1033795119}]},"public_addr":"172.21.15.12:6808/1033795119","cluster_addr":"172.21.15.12:6809/1033795119","heartbeat_back_addr":"172.21.15.12:6811/1033795119","heartbeat_front_addr":"172.21.15.12:6810/1033795119","state":["exists","up"]},{"osd":2,"uuid":"919a351e-ecee-4e06-bff1-ae68c80d320f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6800","nonce":2135679025}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6801","nonce":2135679025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6803","nonce":2135679025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6802","nonce":2135679025}]},"public_addr":"172.21.15.12:6800/2135679025","cluster_addr":"172.21.15.12:6801/2135679025","heartbeat_back_addr":"172.21.15.12:6803/2135679025","heartbeat_front_addr":"172.21.15.12:6802/2135679025","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.109010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:35.949029+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.023975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T04:47:42.268 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-16T04:47:42.268 INFO:tasks.ceph:Creating RBD pool 2024-09-16T04:47:42.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-16T04:47:43.315 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'rbd' created 2024-09-16T04:47:43.340 DEBUG:teuthology.orchestra.run.smithi012:> rbd --cluster ceph pool init rbd 2024-09-16T04:47:46.372 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T04:47:46.372 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T04:47:46.372 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T04:47:46.372 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T04:47:46.373 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T04:47:46.728 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:46.746 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":4,"flags":0,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6812","nonce":4250845417}]},"active_addr":"172.21.15.12:6812/4250845417","active_change":"2024-09-16T04:47:32.232380+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3067517447}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":2304454583}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":32572391}]}]} 2024-09-16T04:47:46.748 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T04:47:46.748 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T04:47:46.748 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T04:47:47.042 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:47.042 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","created":"2024-09-16T04:47:27.160076+0000","modified":"2024-09-16T04:47:46.326414+0000","last_up_change":"2024-09-16T04:47:38.678442+0000","last_in_change":"2024-09-16T04:47:28.313884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-16T04:47:42.612259+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7fa4b8a1-d218-47cb-adc4-469674c829fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6804","nonce":1112855623}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6805","nonce":1112855623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6807","nonce":1112855623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6806","nonce":1112855623}]},"public_addr":"172.21.15.12:6804/1112855623","cluster_addr":"172.21.15.12:6805/1112855623","heartbeat_back_addr":"172.21.15.12:6807/1112855623","heartbeat_front_addr":"172.21.15.12:6806/1112855623","state":["exists","up"]},{"osd":1,"uuid":"7234f3ae-2cf3-4dd4-8c21-4f170976dbf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6808","nonce":1033795119}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6809","nonce":1033795119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6811","nonce":1033795119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6810","nonce":1033795119}]},"public_addr":"172.21.15.12:6808/1033795119","cluster_addr":"172.21.15.12:6809/1033795119","heartbeat_back_addr":"172.21.15.12:6811/1033795119","heartbeat_front_addr":"172.21.15.12:6810/1033795119","state":["exists","up"]},{"osd":2,"uuid":"919a351e-ecee-4e06-bff1-ae68c80d320f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6800","nonce":2135679025}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6801","nonce":2135679025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6803","nonce":2135679025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6802","nonce":2135679025}]},"public_addr":"172.21.15.12:6800/2135679025","cluster_addr":"172.21.15.12:6801/2135679025","heartbeat_back_addr":"172.21.15.12:6803/2135679025","heartbeat_front_addr":"172.21.15.12:6802/2135679025","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.109010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:35.949029+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.023975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T04:47:47.060 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T04:47:47.061 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T04:47:47.350 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:47.350 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","created":"2024-09-16T04:47:27.160076+0000","modified":"2024-09-16T04:47:46.326414+0000","last_up_change":"2024-09-16T04:47:38.678442+0000","last_in_change":"2024-09-16T04:47:28.313884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-16T04:47:42.612259+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7fa4b8a1-d218-47cb-adc4-469674c829fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6804","nonce":1112855623}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6805","nonce":1112855623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6807","nonce":1112855623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6806","nonce":1112855623}]},"public_addr":"172.21.15.12:6804/1112855623","cluster_addr":"172.21.15.12:6805/1112855623","heartbeat_back_addr":"172.21.15.12:6807/1112855623","heartbeat_front_addr":"172.21.15.12:6806/1112855623","state":["exists","up"]},{"osd":1,"uuid":"7234f3ae-2cf3-4dd4-8c21-4f170976dbf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6808","nonce":1033795119}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6809","nonce":1033795119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6811","nonce":1033795119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6810","nonce":1033795119}]},"public_addr":"172.21.15.12:6808/1033795119","cluster_addr":"172.21.15.12:6809/1033795119","heartbeat_back_addr":"172.21.15.12:6811/1033795119","heartbeat_front_addr":"172.21.15.12:6810/1033795119","state":["exists","up"]},{"osd":2,"uuid":"919a351e-ecee-4e06-bff1-ae68c80d320f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6800","nonce":2135679025}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6801","nonce":2135679025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6803","nonce":2135679025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6802","nonce":2135679025}]},"public_addr":"172.21.15.12:6800/2135679025","cluster_addr":"172.21.15.12:6801/2135679025","heartbeat_back_addr":"172.21.15.12:6803/2135679025","heartbeat_front_addr":"172.21.15.12:6802/2135679025","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.109010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:35.949029+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.023975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T04:47:47.367 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T04:47:47.368 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T04:47:47.368 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T04:47:47.515 INFO:teuthology.orchestra.run.smithi012.stdout:38654705668 2024-09-16T04:47:47.515 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T04:47:47.519 INFO:teuthology.orchestra.run.smithi012.stdout:34359738371 2024-09-16T04:47:47.519 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:47:47.526 INFO:teuthology.orchestra.run.smithi012.stdout:38654705667 2024-09-16T04:47:47.527 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T04:47:47.796 INFO:teuthology.orchestra.run.smithi012.stdout:38654705667 2024-09-16T04:47:47.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.2 2024-09-16T04:47:47.850 INFO:teuthology.orchestra.run.smithi012.stdout:38654705666 2024-09-16T04:47:47.970 INFO:teuthology.orchestra.run.smithi012.stdout:34359738370 2024-09-16T04:47:47.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.1 2024-09-16T04:47:47.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738370 for osd.0 2024-09-16T04:47:48.831 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T04:47:48.972 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T04:47:48.973 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:47:49.146 INFO:teuthology.orchestra.run.smithi012.stdout:38654705668 2024-09-16T04:47:49.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.2 2024-09-16T04:47:49.163 DEBUG:teuthology.parallel:result is None 2024-09-16T04:47:49.259 INFO:teuthology.orchestra.run.smithi012.stdout:38654705667 2024-09-16T04:47:49.259 INFO:teuthology.orchestra.run.smithi012.stdout:34359738371 2024-09-16T04:47:49.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705667 for osd.1 2024-09-16T04:47:49.278 DEBUG:teuthology.parallel:result is None 2024-09-16T04:47:49.279 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738371 for osd.0 2024-09-16T04:47:49.279 DEBUG:teuthology.parallel:result is None 2024-09-16T04:47:49.279 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T04:47:49.279 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:47:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:49.555 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:47:49.571 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-16T04:47:48.247080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81460,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034060,"statfs":{"total":289910292480,"available":289826877440,"internally_reserved":0,"allocated":376832,"data_stored":85598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":82963794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.343606+0000","last_change":"2024-09-16T04:47:46.344104+0000","last_active":"2024-09-16T04:47:46.343606+0000","last_peered":"2024-09-16T04:47:46.343606+0000","last_clean":"2024-09-16T04:47:46.343606+0000","last_became_active":"2024-09-16T04:47:44.336686+0000","last_became_peered":"2024-09-16T04:47:44.336686+0000","last_unstale":"2024-09-16T04:47:46.343606+0000","last_undegraded":"2024-09-16T04:47:46.343606+0000","last_fullsized":"2024-09-16T04:47:46.343606+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:57.116372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.342039+0000","last_change":"2024-09-16T04:47:46.342646+0000","last_active":"2024-09-16T04:47:46.342039+0000","last_peered":"2024-09-16T04:47:46.342039+0000","last_clean":"2024-09-16T04:47:46.342039+0000","last_became_active":"2024-09-16T04:47:44.336027+0000","last_became_peered":"2024-09-16T04:47:44.336027+0000","last_unstale":"2024-09-16T04:47:46.342039+0000","last_undegraded":"2024-09-16T04:47:46.342039+0000","last_fullsized":"2024-09-16T04:47:46.342039+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:33.652853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.339380+0000","last_change":"2024-09-16T04:47:46.339996+0000","last_active":"2024-09-16T04:47:46.339380+0000","last_peered":"2024-09-16T04:47:46.339380+0000","last_clean":"2024-09-16T04:47:46.339380+0000","last_became_active":"2024-09-16T04:47:44.336039+0000","last_became_peered":"2024-09-16T04:47:44.336039+0000","last_unstale":"2024-09-16T04:47:46.339380+0000","last_undegraded":"2024-09-16T04:47:46.339380+0000","last_fullsized":"2024-09-16T04:47:46.339380+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:13.015225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011836539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.343059+0000","last_change":"2024-09-16T04:47:46.343554+0000","last_active":"2024-09-16T04:47:46.343059+0000","last_peered":"2024-09-16T04:47:46.343059+0000","last_clean":"2024-09-16T04:47:46.343059+0000","last_became_active":"2024-09-16T04:47:44.336016+0000","last_became_peered":"2024-09-16T04:47:44.336016+0000","last_unstale":"2024-09-16T04:47:46.343059+0000","last_undegraded":"2024-09-16T04:47:46.343059+0000","last_fullsized":"2024-09-16T04:47:46.343059+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:01.791016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.342899+0000","last_change":"2024-09-16T04:47:46.343463+0000","last_active":"2024-09-16T04:47:46.342899+0000","last_peered":"2024-09-16T04:47:46.342899+0000","last_clean":"2024-09-16T04:47:46.342899+0000","last_became_active":"2024-09-16T04:47:44.337472+0000","last_became_peered":"2024-09-16T04:47:44.337472+0000","last_unstale":"2024-09-16T04:47:46.342899+0000","last_undegraded":"2024-09-16T04:47:46.342899+0000","last_fullsized":"2024-09-16T04:47:46.342899+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:32.434431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.350300+0000","last_change":"2024-09-16T04:47:46.341519+0000","last_active":"2024-09-16T04:47:46.350300+0000","last_peered":"2024-09-16T04:47:46.350300+0000","last_clean":"2024-09-16T04:47:46.350300+0000","last_became_active":"2024-09-16T04:47:44.337671+0000","last_became_peered":"2024-09-16T04:47:44.337671+0000","last_unstale":"2024-09-16T04:47:46.350300+0000","last_undegraded":"2024-09-16T04:47:46.350300+0000","last_fullsized":"2024-09-16T04:47:46.350300+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.766406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.339234+0000","last_change":"2024-09-16T04:47:46.339968+0000","last_active":"2024-09-16T04:47:46.339234+0000","last_peered":"2024-09-16T04:47:46.339234+0000","last_clean":"2024-09-16T04:47:46.339234+0000","last_became_active":"2024-09-16T04:47:44.336395+0000","last_became_peered":"2024-09-16T04:47:44.336395+0000","last_unstale":"2024-09-16T04:47:46.339234+0000","last_undegraded":"2024-09-16T04:47:46.339234+0000","last_fullsized":"2024-09-16T04:47:46.339234+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:32.638044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001182974,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.342194+0000","last_change":"2024-09-16T04:47:46.342776+0000","last_active":"2024-09-16T04:47:46.342194+0000","last_peered":"2024-09-16T04:47:46.342194+0000","last_clean":"2024-09-16T04:47:46.342194+0000","last_became_active":"2024-09-16T04:47:44.339613+0000","last_became_peered":"2024-09-16T04:47:44.339613+0000","last_unstale":"2024-09-16T04:47:46.342194+0000","last_undegraded":"2024-09-16T04:47:46.342194+0000","last_fullsized":"2024-09-16T04:47:46.342194+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:25.523051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":122880,"data_stored":28520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":28539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":126976,"data_stored":28539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:47:49.572 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:47:49.858 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:49.859 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:47:49.875 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-16T04:47:48.247080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81460,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034060,"statfs":{"total":289910292480,"available":289826877440,"internally_reserved":0,"allocated":376832,"data_stored":85598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":82963794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.343606+0000","last_change":"2024-09-16T04:47:46.344104+0000","last_active":"2024-09-16T04:47:46.343606+0000","last_peered":"2024-09-16T04:47:46.343606+0000","last_clean":"2024-09-16T04:47:46.343606+0000","last_became_active":"2024-09-16T04:47:44.336686+0000","last_became_peered":"2024-09-16T04:47:44.336686+0000","last_unstale":"2024-09-16T04:47:46.343606+0000","last_undegraded":"2024-09-16T04:47:46.343606+0000","last_fullsized":"2024-09-16T04:47:46.343606+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:57.116372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.342039+0000","last_change":"2024-09-16T04:47:46.342646+0000","last_active":"2024-09-16T04:47:46.342039+0000","last_peered":"2024-09-16T04:47:46.342039+0000","last_clean":"2024-09-16T04:47:46.342039+0000","last_became_active":"2024-09-16T04:47:44.336027+0000","last_became_peered":"2024-09-16T04:47:44.336027+0000","last_unstale":"2024-09-16T04:47:46.342039+0000","last_undegraded":"2024-09-16T04:47:46.342039+0000","last_fullsized":"2024-09-16T04:47:46.342039+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:33.652853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.339380+0000","last_change":"2024-09-16T04:47:46.339996+0000","last_active":"2024-09-16T04:47:46.339380+0000","last_peered":"2024-09-16T04:47:46.339380+0000","last_clean":"2024-09-16T04:47:46.339380+0000","last_became_active":"2024-09-16T04:47:44.336039+0000","last_became_peered":"2024-09-16T04:47:44.336039+0000","last_unstale":"2024-09-16T04:47:46.339380+0000","last_undegraded":"2024-09-16T04:47:46.339380+0000","last_fullsized":"2024-09-16T04:47:46.339380+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:13.015225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011836539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.343059+0000","last_change":"2024-09-16T04:47:46.343554+0000","last_active":"2024-09-16T04:47:46.343059+0000","last_peered":"2024-09-16T04:47:46.343059+0000","last_clean":"2024-09-16T04:47:46.343059+0000","last_became_active":"2024-09-16T04:47:44.336016+0000","last_became_peered":"2024-09-16T04:47:44.336016+0000","last_unstale":"2024-09-16T04:47:46.343059+0000","last_undegraded":"2024-09-16T04:47:46.343059+0000","last_fullsized":"2024-09-16T04:47:46.343059+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:01.791016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.342899+0000","last_change":"2024-09-16T04:47:46.343463+0000","last_active":"2024-09-16T04:47:46.342899+0000","last_peered":"2024-09-16T04:47:46.342899+0000","last_clean":"2024-09-16T04:47:46.342899+0000","last_became_active":"2024-09-16T04:47:44.337472+0000","last_became_peered":"2024-09-16T04:47:44.337472+0000","last_unstale":"2024-09-16T04:47:46.342899+0000","last_undegraded":"2024-09-16T04:47:46.342899+0000","last_fullsized":"2024-09-16T04:47:46.342899+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:32.434431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.350300+0000","last_change":"2024-09-16T04:47:46.341519+0000","last_active":"2024-09-16T04:47:46.350300+0000","last_peered":"2024-09-16T04:47:46.350300+0000","last_clean":"2024-09-16T04:47:46.350300+0000","last_became_active":"2024-09-16T04:47:44.337671+0000","last_became_peered":"2024-09-16T04:47:44.337671+0000","last_unstale":"2024-09-16T04:47:46.350300+0000","last_undegraded":"2024-09-16T04:47:46.350300+0000","last_fullsized":"2024-09-16T04:47:46.350300+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.766406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.339234+0000","last_change":"2024-09-16T04:47:46.339968+0000","last_active":"2024-09-16T04:47:46.339234+0000","last_peered":"2024-09-16T04:47:46.339234+0000","last_clean":"2024-09-16T04:47:46.339234+0000","last_became_active":"2024-09-16T04:47:44.336395+0000","last_became_peered":"2024-09-16T04:47:44.336395+0000","last_unstale":"2024-09-16T04:47:46.339234+0000","last_undegraded":"2024-09-16T04:47:46.339234+0000","last_fullsized":"2024-09-16T04:47:46.339234+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:32.638044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001182974,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.342194+0000","last_change":"2024-09-16T04:47:46.342776+0000","last_active":"2024-09-16T04:47:46.342194+0000","last_peered":"2024-09-16T04:47:46.342194+0000","last_clean":"2024-09-16T04:47:46.342194+0000","last_became_active":"2024-09-16T04:47:44.339613+0000","last_became_peered":"2024-09-16T04:47:44.339613+0000","last_unstale":"2024-09-16T04:47:46.342194+0000","last_undegraded":"2024-09-16T04:47:46.342194+0000","last_fullsized":"2024-09-16T04:47:46.342194+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:25.523051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":122880,"data_stored":28520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":28539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":126976,"data_stored":28539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:47:49.876 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T04:47:49.876 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T04:47:49.876 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T04:47:49.876 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T04:47:50.202 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:50.202 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T04:47:50.220 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T04:47:50.220 INFO:teuthology.run_tasks:Running task rep_lost_unfound_delete... 2024-09-16T04:47:50.247 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T04:47:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:50.539 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","created":"2024-09-16T04:47:27.160076+0000","modified":"2024-09-16T04:47:46.326414+0000","last_up_change":"2024-09-16T04:47:38.678442+0000","last_in_change":"2024-09-16T04:47:28.313884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-16T04:47:42.612259+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7fa4b8a1-d218-47cb-adc4-469674c829fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6804","nonce":1112855623}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6805","nonce":1112855623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6807","nonce":1112855623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6806","nonce":1112855623}]},"public_addr":"172.21.15.12:6804/1112855623","cluster_addr":"172.21.15.12:6805/1112855623","heartbeat_back_addr":"172.21.15.12:6807/1112855623","heartbeat_front_addr":"172.21.15.12:6806/1112855623","state":["exists","up"]},{"osd":1,"uuid":"7234f3ae-2cf3-4dd4-8c21-4f170976dbf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6808","nonce":1033795119}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6809","nonce":1033795119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6811","nonce":1033795119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6810","nonce":1033795119}]},"public_addr":"172.21.15.12:6808/1033795119","cluster_addr":"172.21.15.12:6809/1033795119","heartbeat_back_addr":"172.21.15.12:6811/1033795119","heartbeat_front_addr":"172.21.15.12:6810/1033795119","state":["exists","up"]},{"osd":2,"uuid":"919a351e-ecee-4e06-bff1-ae68c80d320f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6800","nonce":2135679025}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6801","nonce":2135679025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6803","nonce":2135679025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6802","nonce":2135679025}]},"public_addr":"172.21.15.12:6800/2135679025","cluster_addr":"172.21.15.12:6801/2135679025","heartbeat_back_addr":"172.21.15.12:6803/2135679025","heartbeat_front_addr":"172.21.15.12:6802/2135679025","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.109010+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:35.949029+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.023975+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T04:47:50.558 INFO:tasks.rep_lost_unfound_delete.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-16T04:47:42.612259+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 14, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T04:47:50.558 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-16T04:47:50.858 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 8 2024-09-16T04:47:50.876 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-16T04:47:51.180 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 14 2024-09-16T04:47:51.180 INFO:teuthology.orchestra.run.smithi012.stdout:fsid b9a92eba-c518-4afa-ae65-ed2825dd0c06 2024-09-16T04:47:51.180 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-09-16T04:47:27.160076+0000 2024-09-16T04:47:51.180 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-09-16T04:47:46.326414+0000 2024-09-16T04:47:51.180 INFO:teuthology.orchestra.run.smithi012.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T04:47:51.180 INFO:teuthology.orchestra.run.smithi012.stdout:crush_version 4 2024-09-16T04:47:51.180 INFO:teuthology.orchestra.run.smithi012.stdout:full_ratio 0.95 2024-09-16T04:47:51.180 INFO:teuthology.orchestra.run.smithi012.stdout:backfillfull_ratio 0.9 2024-09-16T04:47:51.180 INFO:teuthology.orchestra.run.smithi012.stdout:nearfull_ratio 0.85 2024-09-16T04:47:51.180 INFO:teuthology.orchestra.run.smithi012.stdout:require_min_compat_client luminous 2024-09-16T04:47:51.181 INFO:teuthology.orchestra.run.smithi012.stdout:min_compat_client jewel 2024-09-16T04:47:51.181 INFO:teuthology.orchestra.run.smithi012.stdout:require_osd_release squid 2024-09-16T04:47:51.181 INFO:teuthology.orchestra.run.smithi012.stdout:stretch_mode_enabled false 2024-09-16T04:47:51.181 INFO:teuthology.orchestra.run.smithi012.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 14 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.88 2024-09-16T04:47:51.181 INFO:teuthology.orchestra.run.smithi012.stdout: removed_snaps_queue [2~1] 2024-09-16T04:47:51.181 INFO:teuthology.orchestra.run.smithi012.stdout:max_osd 3 2024-09-16T04:47:51.181 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.12:6804/1112855623 v1:172.21.15.12:6805/1112855623 exists,up 7fa4b8a1-d218-47cb-adc4-469674c829fc 2024-09-16T04:47:51.181 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.12:6808/1033795119 v1:172.21.15.12:6809/1033795119 exists,up 7234f3ae-2cf3-4dd4-8c21-4f170976dbf4 2024-09-16T04:47:51.181 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.12:6800/2135679025 v1:172.21.15.12:6801/2135679025 exists,up 919a351e-ecee-4e06-bff1-ae68c80d320f 2024-09-16T04:47:51.198 INFO:tasks.rep_lost_unfound_delete.ceph_manager:['osd.0 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.12:6804/1112855623 v1:172.21.15.12:6805/1112855623 exists,up 7fa4b8a1-d218-47cb-adc4-469674c829fc', 'osd.1 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.12:6808/1033795119 v1:172.21.15.12:6809/1033795119 exists,up 7234f3ae-2cf3-4dd4-8c21-4f170976dbf4', 'osd.2 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.12:6800/2135679025 v1:172.21.15.12:6801/2135679025 exists,up 919a351e-ecee-4e06-bff1-ae68c80d320f'] 2024-09-16T04:47:51.198 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T04:47:51.198 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T04:47:51.199 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T04:47:51.348 INFO:teuthology.orchestra.run.smithi012.stdout:38654705670 2024-09-16T04:47:51.349 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T04:47:51.351 INFO:teuthology.orchestra.run.smithi012.stdout:38654705669 2024-09-16T04:47:51.351 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T04:47:51.360 INFO:teuthology.orchestra.run.smithi012.stdout:34359738373 2024-09-16T04:47:51.361 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:47:51.661 INFO:teuthology.orchestra.run.smithi012.stdout:38654705669 2024-09-16T04:47:51.678 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705670 got 38654705669 for osd.2 2024-09-16T04:47:51.688 INFO:teuthology.orchestra.run.smithi012.stdout:34359738372 2024-09-16T04:47:51.701 INFO:teuthology.orchestra.run.smithi012.stdout:38654705668 2024-09-16T04:47:51.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738373 got 34359738372 for osd.0 2024-09-16T04:47:51.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705669 got 38654705668 for osd.1 2024-09-16T04:47:52.679 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T04:47:52.705 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:47:52.720 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T04:47:52.992 INFO:teuthology.orchestra.run.smithi012.stdout:38654705669 2024-09-16T04:47:53.009 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705670 got 38654705669 for osd.2 2024-09-16T04:47:53.021 INFO:teuthology.orchestra.run.smithi012.stdout:34359738372 2024-09-16T04:47:53.026 INFO:teuthology.orchestra.run.smithi012.stdout:38654705668 2024-09-16T04:47:53.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738373 got 34359738372 for osd.0 2024-09-16T04:47:53.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705669 got 38654705668 for osd.1 2024-09-16T04:47:54.010 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T04:47:54.040 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:47:54.049 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T04:47:54.324 INFO:teuthology.orchestra.run.smithi012.stdout:38654705670 2024-09-16T04:47:54.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705670 got 38654705670 for osd.2 2024-09-16T04:47:54.341 DEBUG:teuthology.parallel:result is None 2024-09-16T04:47:54.351 INFO:teuthology.orchestra.run.smithi012.stdout:34359738373 2024-09-16T04:47:54.354 INFO:teuthology.orchestra.run.smithi012.stdout:38654705669 2024-09-16T04:47:54.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738373 got 34359738373 for osd.0 2024-09-16T04:47:54.373 DEBUG:teuthology.parallel:result is None 2024-09-16T04:47:54.376 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705669 got 38654705669 for osd.1 2024-09-16T04:47:54.376 DEBUG:teuthology.parallel:result is None 2024-09-16T04:47:54.376 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-16T04:47:54.376 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:47:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:54.699 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:47:54.715 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-16T04:47:54.248581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81460,"kb_used_data":376,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034060,"statfs":{"total":289910292480,"available":289826877440,"internally_reserved":0,"allocated":385024,"data_stored":87988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":82963794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001500"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.343606+0000","last_change":"2024-09-16T04:47:46.344104+0000","last_active":"2024-09-16T04:47:46.343606+0000","last_peered":"2024-09-16T04:47:46.343606+0000","last_clean":"2024-09-16T04:47:46.343606+0000","last_became_active":"2024-09-16T04:47:44.336686+0000","last_became_peered":"2024-09-16T04:47:44.336686+0000","last_unstale":"2024-09-16T04:47:46.343606+0000","last_undegraded":"2024-09-16T04:47:46.343606+0000","last_fullsized":"2024-09-16T04:47:46.343606+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:57.116372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.342039+0000","last_change":"2024-09-16T04:47:46.342646+0000","last_active":"2024-09-16T04:47:46.342039+0000","last_peered":"2024-09-16T04:47:46.342039+0000","last_clean":"2024-09-16T04:47:46.342039+0000","last_became_active":"2024-09-16T04:47:44.336027+0000","last_became_peered":"2024-09-16T04:47:44.336027+0000","last_unstale":"2024-09-16T04:47:46.342039+0000","last_undegraded":"2024-09-16T04:47:46.342039+0000","last_fullsized":"2024-09-16T04:47:46.342039+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:33.652853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.339380+0000","last_change":"2024-09-16T04:47:46.339996+0000","last_active":"2024-09-16T04:47:46.339380+0000","last_peered":"2024-09-16T04:47:46.339380+0000","last_clean":"2024-09-16T04:47:46.339380+0000","last_became_active":"2024-09-16T04:47:44.336039+0000","last_became_peered":"2024-09-16T04:47:44.336039+0000","last_unstale":"2024-09-16T04:47:46.339380+0000","last_undegraded":"2024-09-16T04:47:46.339380+0000","last_fullsized":"2024-09-16T04:47:46.339380+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:13.015225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011836539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.343059+0000","last_change":"2024-09-16T04:47:46.343554+0000","last_active":"2024-09-16T04:47:46.343059+0000","last_peered":"2024-09-16T04:47:46.343059+0000","last_clean":"2024-09-16T04:47:46.343059+0000","last_became_active":"2024-09-16T04:47:44.336016+0000","last_became_peered":"2024-09-16T04:47:44.336016+0000","last_unstale":"2024-09-16T04:47:46.343059+0000","last_undegraded":"2024-09-16T04:47:46.343059+0000","last_fullsized":"2024-09-16T04:47:46.343059+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:01.791016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.342899+0000","last_change":"2024-09-16T04:47:46.343463+0000","last_active":"2024-09-16T04:47:46.342899+0000","last_peered":"2024-09-16T04:47:46.342899+0000","last_clean":"2024-09-16T04:47:46.342899+0000","last_became_active":"2024-09-16T04:47:44.337472+0000","last_became_peered":"2024-09-16T04:47:44.337472+0000","last_unstale":"2024-09-16T04:47:46.342899+0000","last_undegraded":"2024-09-16T04:47:46.342899+0000","last_fullsized":"2024-09-16T04:47:46.342899+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:32.434431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T04:47:52.102598+0000","last_change":"2024-09-16T04:47:46.341519+0000","last_active":"2024-09-16T04:47:52.102598+0000","last_peered":"2024-09-16T04:47:52.102598+0000","last_clean":"2024-09-16T04:47:52.102598+0000","last_became_active":"2024-09-16T04:47:44.337671+0000","last_became_peered":"2024-09-16T04:47:44.337671+0000","last_unstale":"2024-09-16T04:47:52.102598+0000","last_undegraded":"2024-09-16T04:47:52.102598+0000","last_fullsized":"2024-09-16T04:47:52.102598+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.766406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.339234+0000","last_change":"2024-09-16T04:47:46.339968+0000","last_active":"2024-09-16T04:47:46.339234+0000","last_peered":"2024-09-16T04:47:46.339234+0000","last_clean":"2024-09-16T04:47:46.339234+0000","last_became_active":"2024-09-16T04:47:44.336395+0000","last_became_peered":"2024-09-16T04:47:44.336395+0000","last_unstale":"2024-09-16T04:47:46.339234+0000","last_undegraded":"2024-09-16T04:47:46.339234+0000","last_fullsized":"2024-09-16T04:47:46.339234+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:32.638044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001182974,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.342194+0000","last_change":"2024-09-16T04:47:46.342776+0000","last_active":"2024-09-16T04:47:46.342194+0000","last_peered":"2024-09-16T04:47:46.342194+0000","last_clean":"2024-09-16T04:47:46.342194+0000","last_became_active":"2024-09-16T04:47:44.339613+0000","last_became_peered":"2024-09-16T04:47:44.339613+0000","last_unstale":"2024-09-16T04:47:46.342194+0000","last_undegraded":"2024-09-16T04:47:46.342194+0000","last_fullsized":"2024-09-16T04:47:46.342194+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:25.523051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":122880,"data_stored":28520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":28539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":135168,"data_stored":30929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:47:54.716 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:47:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:47:55.003 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:47:55.018 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-16T04:47:54.248581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81460,"kb_used_data":376,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034060,"statfs":{"total":289910292480,"available":289826877440,"internally_reserved":0,"allocated":385024,"data_stored":87988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":82963794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001500"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.343606+0000","last_change":"2024-09-16T04:47:46.344104+0000","last_active":"2024-09-16T04:47:46.343606+0000","last_peered":"2024-09-16T04:47:46.343606+0000","last_clean":"2024-09-16T04:47:46.343606+0000","last_became_active":"2024-09-16T04:47:44.336686+0000","last_became_peered":"2024-09-16T04:47:44.336686+0000","last_unstale":"2024-09-16T04:47:46.343606+0000","last_undegraded":"2024-09-16T04:47:46.343606+0000","last_fullsized":"2024-09-16T04:47:46.343606+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:57.116372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.342039+0000","last_change":"2024-09-16T04:47:46.342646+0000","last_active":"2024-09-16T04:47:46.342039+0000","last_peered":"2024-09-16T04:47:46.342039+0000","last_clean":"2024-09-16T04:47:46.342039+0000","last_became_active":"2024-09-16T04:47:44.336027+0000","last_became_peered":"2024-09-16T04:47:44.336027+0000","last_unstale":"2024-09-16T04:47:46.342039+0000","last_undegraded":"2024-09-16T04:47:46.342039+0000","last_fullsized":"2024-09-16T04:47:46.342039+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:33.652853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.339380+0000","last_change":"2024-09-16T04:47:46.339996+0000","last_active":"2024-09-16T04:47:46.339380+0000","last_peered":"2024-09-16T04:47:46.339380+0000","last_clean":"2024-09-16T04:47:46.339380+0000","last_became_active":"2024-09-16T04:47:44.336039+0000","last_became_peered":"2024-09-16T04:47:44.336039+0000","last_unstale":"2024-09-16T04:47:46.339380+0000","last_undegraded":"2024-09-16T04:47:46.339380+0000","last_fullsized":"2024-09-16T04:47:46.339380+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:13.015225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011836539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.343059+0000","last_change":"2024-09-16T04:47:46.343554+0000","last_active":"2024-09-16T04:47:46.343059+0000","last_peered":"2024-09-16T04:47:46.343059+0000","last_clean":"2024-09-16T04:47:46.343059+0000","last_became_active":"2024-09-16T04:47:44.336016+0000","last_became_peered":"2024-09-16T04:47:44.336016+0000","last_unstale":"2024-09-16T04:47:46.343059+0000","last_undegraded":"2024-09-16T04:47:46.343059+0000","last_fullsized":"2024-09-16T04:47:46.343059+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:01.791016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.342899+0000","last_change":"2024-09-16T04:47:46.343463+0000","last_active":"2024-09-16T04:47:46.342899+0000","last_peered":"2024-09-16T04:47:46.342899+0000","last_clean":"2024-09-16T04:47:46.342899+0000","last_became_active":"2024-09-16T04:47:44.337472+0000","last_became_peered":"2024-09-16T04:47:44.337472+0000","last_unstale":"2024-09-16T04:47:46.342899+0000","last_undegraded":"2024-09-16T04:47:46.342899+0000","last_fullsized":"2024-09-16T04:47:46.342899+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:00:32.434431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T04:47:52.102598+0000","last_change":"2024-09-16T04:47:46.341519+0000","last_active":"2024-09-16T04:47:52.102598+0000","last_peered":"2024-09-16T04:47:52.102598+0000","last_clean":"2024-09-16T04:47:52.102598+0000","last_became_active":"2024-09-16T04:47:44.337671+0000","last_became_peered":"2024-09-16T04:47:44.337671+0000","last_unstale":"2024-09-16T04:47:52.102598+0000","last_undegraded":"2024-09-16T04:47:52.102598+0000","last_fullsized":"2024-09-16T04:47:52.102598+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.766406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.339234+0000","last_change":"2024-09-16T04:47:46.339968+0000","last_active":"2024-09-16T04:47:46.339234+0000","last_peered":"2024-09-16T04:47:46.339234+0000","last_clean":"2024-09-16T04:47:46.339234+0000","last_became_active":"2024-09-16T04:47:44.336395+0000","last_became_peered":"2024-09-16T04:47:44.336395+0000","last_unstale":"2024-09-16T04:47:46.339234+0000","last_undegraded":"2024-09-16T04:47:46.339234+0000","last_fullsized":"2024-09-16T04:47:46.339234+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:32.638044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001182974,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T04:47:46.342194+0000","last_change":"2024-09-16T04:47:46.342776+0000","last_active":"2024-09-16T04:47:46.342194+0000","last_peered":"2024-09-16T04:47:46.342194+0000","last_clean":"2024-09-16T04:47:46.342194+0000","last_became_active":"2024-09-16T04:47:44.339613+0000","last_became_peered":"2024-09-16T04:47:44.339613+0000","last_unstale":"2024-09-16T04:47:46.342194+0000","last_undegraded":"2024-09-16T04:47:46.342194+0000","last_fullsized":"2024-09-16T04:47:46.342194+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:25.523051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":122880,"data_stored":28520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":28539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":135168,"data_stored":30929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:47:55.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-16T04:47:55.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:creating pool_name unfounddel_pool 2024-09-16T04:47:55.020 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unfounddel_pool 16 2024-09-16T04:47:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'unfounddel_pool' created 2024-09-16T04:47:55.414 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unfounddel_pool rados --yes-i-really-mean-it || true 2024-09-16T04:47:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:enabled application 'rados' on pool 'unfounddel_pool' 2024-09-16T04:47:57.436 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T04:47:57.436 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T04:47:57.501 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T04:47:57.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-16T04:47:57.817 INFO:tasks.ceph.mon.c.smithi012.stderr:2024-09-16T04:47:57.813+0000 7fbe3b864640 -1 mon.c@2(leader).osd e18 definitely_dead 0 2024-09-16T04:47:58.420 INFO:teuthology.orchestra.run.smithi012.stderr:marked down osd.2. 2024-09-16T04:47:58.447 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-16T04:47:58.750 INFO:tasks.ceph.mon.c.smithi012.stderr:2024-09-16T04:47:58.745+0000 7fbe3b864640 -1 mon.c@2(leader).osd e19 definitely_dead 0 2024-09-16T04:47:59.431 INFO:teuthology.orchestra.run.smithi012.stderr:marked out osd.2. 2024-09-16T04:47:59.455 INFO:tasks.util.rados:rados -p unfounddel_pool put dummy /etc/fstab 2024-09-16T04:47:59.455 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put dummy /etc/fstab 2024-09-16T04:47:59.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T04:47:59.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T04:47:59.696 INFO:teuthology.orchestra.run.smithi012.stdout:34359738376 2024-09-16T04:47:59.696 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:47:59.699 INFO:teuthology.orchestra.run.smithi012.stdout:38654705672 2024-09-16T04:47:59.699 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T04:47:59.984 INFO:teuthology.orchestra.run.smithi012.stdout:34359738374 2024-09-16T04:48:00.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738376 got 34359738374 for osd.0 2024-09-16T04:48:00.030 INFO:teuthology.orchestra.run.smithi012.stdout:38654705670 2024-09-16T04:48:00.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705672 got 38654705670 for osd.1 2024-09-16T04:48:01.002 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:48:01.047 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T04:48:01.318 INFO:teuthology.orchestra.run.smithi012.stdout:34359738376 2024-09-16T04:48:01.336 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738376 got 34359738376 for osd.0 2024-09-16T04:48:01.336 DEBUG:teuthology.parallel:result is None 2024-09-16T04:48:01.340 INFO:teuthology.orchestra.run.smithi012.stdout:38654705672 2024-09-16T04:48:01.357 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705672 got 38654705672 for osd.1 2024-09-16T04:48:01.357 DEBUG:teuthology.parallel:result is None 2024-09-16T04:48:01.357 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-16T04:48:01.358 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:48:01.690 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:01.691 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:48:01.709 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-16T04:48:00.423839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54392,"kb_used_data":328,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689288,"statfs":{"total":193273528320,"available":193217830912,"internally_reserved":0,"allocated":335872,"data_stored":86216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3188,"internal_metadata":55309196},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035679"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.429370+0000","last_change":"2024-09-16T04:47:56.406884+0000","last_active":"2024-09-16T04:47:59.429370+0000","last_peered":"2024-09-16T04:47:59.429370+0000","last_clean":"2024-09-16T04:47:59.429370+0000","last_became_active":"2024-09-16T04:47:56.405974+0000","last_became_peered":"2024-09-16T04:47:56.405974+0000","last_unstale":"2024-09-16T04:47:59.429370+0000","last_undegraded":"2024-09-16T04:47:59.429370+0000","last_fullsized":"2024-09-16T04:47:59.429370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:04.264841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-16T04:47:59.430410+0000","last_change":"2024-09-16T04:47:58.417096+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.430410+0000","last_undegraded":"2024-09-16T04:47:59.430410+0000","last_fullsized":"2024-09-16T04:47:59.430410+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317395+0000","last_change":"2024-09-16T04:47:58.413540+0000","last_active":"2024-09-16T04:47:58.413540+0000","last_peered":"2024-09-16T04:47:57.396939+0000","last_clean":"2024-09-16T04:47:57.396939+0000","last_became_active":"2024-09-16T04:47:56.410391+0000","last_became_peered":"2024-09-16T04:47:56.410391+0000","last_unstale":"2024-09-16T04:47:59.317395+0000","last_undegraded":"2024-09-16T04:47:59.317395+0000","last_fullsized":"2024-09-16T04:47:59.317395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-16T04:47:59.429521+0000","last_change":"2024-09-16T04:47:58.416847+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.429521+0000","last_undegraded":"2024-09-16T04:47:59.429521+0000","last_fullsized":"2024-09-16T04:47:59.429521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.415158+0000","last_change":"2024-09-16T04:47:58.415158+0000","last_active":"2024-09-16T04:47:58.415158+0000","last_peered":"2024-09-16T04:47:58.315565+0000","last_clean":"2024-09-16T04:47:58.315565+0000","last_became_active":"2024-09-16T04:47:56.401138+0000","last_became_peered":"2024-09-16T04:47:56.401138+0000","last_unstale":"2024-09-16T04:47:58.415158+0000","last_undegraded":"2024-09-16T04:47:58.415158+0000","last_fullsized":"2024-09-16T04:47:58.415158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423488+0000","last_change":"2024-09-16T04:47:56.416639+0000","last_active":"2024-09-16T04:47:59.423488+0000","last_peered":"2024-09-16T04:47:59.423488+0000","last_clean":"2024-09-16T04:47:59.423488+0000","last_became_active":"2024-09-16T04:47:56.416184+0000","last_became_peered":"2024-09-16T04:47:56.416184+0000","last_unstale":"2024-09-16T04:47:59.423488+0000","last_undegraded":"2024-09-16T04:47:59.423488+0000","last_fullsized":"2024-09-16T04:47:59.423488+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:59.275401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423387+0000","last_change":"2024-09-16T04:47:56.416550+0000","last_active":"2024-09-16T04:47:59.423387+0000","last_peered":"2024-09-16T04:47:59.423387+0000","last_clean":"2024-09-16T04:47:59.423387+0000","last_became_active":"2024-09-16T04:47:56.416077+0000","last_became_peered":"2024-09-16T04:47:56.416077+0000","last_unstale":"2024-09-16T04:47:59.423387+0000","last_undegraded":"2024-09-16T04:47:59.423387+0000","last_fullsized":"2024-09-16T04:47:59.423387+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:14.550234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.412604+0000","last_change":"2024-09-16T04:47:58.412604+0000","last_active":"2024-09-16T04:47:58.412604+0000","last_peered":"2024-09-16T04:47:58.315475+0000","last_clean":"2024-09-16T04:47:58.315475+0000","last_became_active":"2024-09-16T04:47:56.401094+0000","last_became_peered":"2024-09-16T04:47:56.401094+0000","last_unstale":"2024-09-16T04:47:58.412604+0000","last_undegraded":"2024-09-16T04:47:58.412604+0000","last_fullsized":"2024-09-16T04:47:58.412604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.422693+0000","last_change":"2024-09-16T04:47:46.342776+0000","last_active":"2024-09-16T04:47:59.422693+0000","last_peered":"2024-09-16T04:47:59.422693+0000","last_clean":"2024-09-16T04:47:59.422693+0000","last_became_active":"2024-09-16T04:47:44.339613+0000","last_became_peered":"2024-09-16T04:47:44.339613+0000","last_unstale":"2024-09-16T04:47:59.422693+0000","last_undegraded":"2024-09-16T04:47:59.422693+0000","last_fullsized":"2024-09-16T04:47:59.422693+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:25.523051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317349+0000","last_change":"2024-09-16T04:47:58.413217+0000","last_active":"2024-09-16T04:47:58.413217+0000","last_peered":"2024-09-16T04:47:57.396117+0000","last_clean":"2024-09-16T04:47:57.396117+0000","last_became_active":"2024-09-16T04:47:56.413251+0000","last_became_peered":"2024-09-16T04:47:56.413251+0000","last_unstale":"2024-09-16T04:47:59.317349+0000","last_undegraded":"2024-09-16T04:47:59.317349+0000","last_fullsized":"2024-09-16T04:47:59.317349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-16T04:47:59.426984+0000","last_change":"2024-09-16T04:47:58.415328+0000","last_active":"2024-09-16T04:47:43.303429+0000","last_peered":"2024-09-16T04:47:43.303429+0000","last_clean":"2024-09-16T04:47:43.303429+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.426984+0000","last_undegraded":"2024-09-16T04:47:59.426984+0000","last_fullsized":"2024-09-16T04:47:59.426984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.427452+0000","last_change":"2024-09-16T04:47:56.406017+0000","last_active":"2024-09-16T04:47:59.427452+0000","last_peered":"2024-09-16T04:47:59.427452+0000","last_clean":"2024-09-16T04:47:59.427452+0000","last_became_active":"2024-09-16T04:47:56.405573+0000","last_became_peered":"2024-09-16T04:47:56.405573+0000","last_unstale":"2024-09-16T04:47:59.427452+0000","last_undegraded":"2024-09-16T04:47:59.427452+0000","last_fullsized":"2024-09-16T04:47:59.427452+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:31.014311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":33,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.427561+0000","last_change":"2024-09-16T04:47:46.341519+0000","last_active":"2024-09-16T04:47:59.427561+0000","last_peered":"2024-09-16T04:47:59.427561+0000","last_clean":"2024-09-16T04:47:59.427561+0000","last_became_active":"2024-09-16T04:47:44.337671+0000","last_became_peered":"2024-09-16T04:47:44.337671+0000","last_unstale":"2024-09-16T04:47:59.427561+0000","last_undegraded":"2024-09-16T04:47:59.427561+0000","last_fullsized":"2024-09-16T04:47:59.427561+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.766406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317288+0000","last_change":"2024-09-16T04:47:58.416259+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.317288+0000","last_undegraded":"2024-09-16T04:47:59.317288+0000","last_fullsized":"2024-09-16T04:47:59.317288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.414973+0000","last_change":"2024-09-16T04:47:58.414973+0000","last_active":"2024-09-16T04:47:58.414973+0000","last_peered":"2024-09-16T04:47:57.395966+0000","last_clean":"2024-09-16T04:47:57.395966+0000","last_became_active":"2024-09-16T04:47:44.337472+0000","last_became_peered":"2024-09-16T04:47:44.337472+0000","last_unstale":"2024-09-16T04:47:58.414973+0000","last_undegraded":"2024-09-16T04:47:58.414973+0000","last_fullsized":"2024-09-16T04:47:58.414973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317242+0000","last_change":"2024-09-16T04:47:58.412562+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.317242+0000","last_undegraded":"2024-09-16T04:47:59.317242+0000","last_fullsized":"2024-09-16T04:47:59.317242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423507+0000","last_change":"2024-09-16T04:47:56.412192+0000","last_active":"2024-09-16T04:47:59.423507+0000","last_peered":"2024-09-16T04:47:59.423507+0000","last_clean":"2024-09-16T04:47:59.423507+0000","last_became_active":"2024-09-16T04:47:56.411398+0000","last_became_peered":"2024-09-16T04:47:56.411398+0000","last_unstale":"2024-09-16T04:47:59.423507+0000","last_undegraded":"2024-09-16T04:47:59.423507+0000","last_fullsized":"2024-09-16T04:47:59.423507+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:58.421611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423619+0000","last_change":"2024-09-16T04:47:46.343554+0000","last_active":"2024-09-16T04:47:59.423619+0000","last_peered":"2024-09-16T04:47:59.423619+0000","last_clean":"2024-09-16T04:47:59.423619+0000","last_became_active":"2024-09-16T04:47:44.336016+0000","last_became_peered":"2024-09-16T04:47:44.336016+0000","last_unstale":"2024-09-16T04:47:59.423619+0000","last_undegraded":"2024-09-16T04:47:59.423619+0000","last_fullsized":"2024-09-16T04:47:59.423619+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:01.791016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423883+0000","last_change":"2024-09-16T04:47:56.416361+0000","last_active":"2024-09-16T04:47:59.423883+0000","last_peered":"2024-09-16T04:47:59.423883+0000","last_clean":"2024-09-16T04:47:59.423883+0000","last_became_active":"2024-09-16T04:47:56.409348+0000","last_became_peered":"2024-09-16T04:47:56.409348+0000","last_unstale":"2024-09-16T04:47:59.423883+0000","last_undegraded":"2024-09-16T04:47:59.423883+0000","last_fullsized":"2024-09-16T04:47:59.423883+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:07.297529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-16T04:47:59.429668+0000","last_change":"2024-09-16T04:47:58.418094+0000","last_active":"2024-09-16T04:47:43.303429+0000","last_peered":"2024-09-16T04:47:43.303429+0000","last_clean":"2024-09-16T04:47:43.303429+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.429668+0000","last_undegraded":"2024-09-16T04:47:59.429668+0000","last_fullsized":"2024-09-16T04:47:59.429668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.424911+0000","last_change":"2024-09-16T04:47:56.410174+0000","last_active":"2024-09-16T04:47:59.424911+0000","last_peered":"2024-09-16T04:47:59.424911+0000","last_clean":"2024-09-16T04:47:59.424911+0000","last_became_active":"2024-09-16T04:47:56.409473+0000","last_became_peered":"2024-09-16T04:47:56.409473+0000","last_unstale":"2024-09-16T04:47:59.424911+0000","last_undegraded":"2024-09-16T04:47:59.424911+0000","last_fullsized":"2024-09-16T04:47:59.424911+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:09.086995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.424116+0000","last_change":"2024-09-16T04:47:46.342646+0000","last_active":"2024-09-16T04:47:59.424116+0000","last_peered":"2024-09-16T04:47:59.424116+0000","last_clean":"2024-09-16T04:47:59.424116+0000","last_became_active":"2024-09-16T04:47:44.336027+0000","last_became_peered":"2024-09-16T04:47:44.336027+0000","last_unstale":"2024-09-16T04:47:59.424116+0000","last_undegraded":"2024-09-16T04:47:59.424116+0000","last_fullsized":"2024-09-16T04:47:59.424116+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:33.652853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":29,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.415940+0000","last_change":"2024-09-16T04:47:58.415940+0000","last_active":"2024-09-16T04:47:58.415940+0000","last_peered":"2024-09-16T04:47:57.396629+0000","last_clean":"2024-09-16T04:47:57.396629+0000","last_became_active":"2024-09-16T04:47:44.336686+0000","last_became_peered":"2024-09-16T04:47:44.336686+0000","last_unstale":"2024-09-16T04:47:58.415940+0000","last_undegraded":"2024-09-16T04:47:58.415940+0000","last_fullsized":"2024-09-16T04:47:58.415940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423505+0000","last_change":"2024-09-16T04:47:56.416536+0000","last_active":"2024-09-16T04:47:59.423505+0000","last_peered":"2024-09-16T04:47:59.423505+0000","last_clean":"2024-09-16T04:47:59.423505+0000","last_became_active":"2024-09-16T04:47:56.416197+0000","last_became_peered":"2024-09-16T04:47:56.416197+0000","last_unstale":"2024-09-16T04:47:59.423505+0000","last_undegraded":"2024-09-16T04:47:59.423505+0000","last_fullsized":"2024-09-16T04:47:59.423505+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:41.783205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":43108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":167936,"data_stored":43108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:48:01.710 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:48:02.001 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:02.002 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:48:02.018 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-16T04:48:00.423839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54392,"kb_used_data":328,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689288,"statfs":{"total":193273528320,"available":193217830912,"internally_reserved":0,"allocated":335872,"data_stored":86216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3188,"internal_metadata":55309196},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035679"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.429370+0000","last_change":"2024-09-16T04:47:56.406884+0000","last_active":"2024-09-16T04:47:59.429370+0000","last_peered":"2024-09-16T04:47:59.429370+0000","last_clean":"2024-09-16T04:47:59.429370+0000","last_became_active":"2024-09-16T04:47:56.405974+0000","last_became_peered":"2024-09-16T04:47:56.405974+0000","last_unstale":"2024-09-16T04:47:59.429370+0000","last_undegraded":"2024-09-16T04:47:59.429370+0000","last_fullsized":"2024-09-16T04:47:59.429370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:04.264841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-16T04:47:59.430410+0000","last_change":"2024-09-16T04:47:58.417096+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.430410+0000","last_undegraded":"2024-09-16T04:47:59.430410+0000","last_fullsized":"2024-09-16T04:47:59.430410+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317395+0000","last_change":"2024-09-16T04:47:58.413540+0000","last_active":"2024-09-16T04:47:58.413540+0000","last_peered":"2024-09-16T04:47:57.396939+0000","last_clean":"2024-09-16T04:47:57.396939+0000","last_became_active":"2024-09-16T04:47:56.410391+0000","last_became_peered":"2024-09-16T04:47:56.410391+0000","last_unstale":"2024-09-16T04:47:59.317395+0000","last_undegraded":"2024-09-16T04:47:59.317395+0000","last_fullsized":"2024-09-16T04:47:59.317395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-16T04:47:59.429521+0000","last_change":"2024-09-16T04:47:58.416847+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.429521+0000","last_undegraded":"2024-09-16T04:47:59.429521+0000","last_fullsized":"2024-09-16T04:47:59.429521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.415158+0000","last_change":"2024-09-16T04:47:58.415158+0000","last_active":"2024-09-16T04:47:58.415158+0000","last_peered":"2024-09-16T04:47:58.315565+0000","last_clean":"2024-09-16T04:47:58.315565+0000","last_became_active":"2024-09-16T04:47:56.401138+0000","last_became_peered":"2024-09-16T04:47:56.401138+0000","last_unstale":"2024-09-16T04:47:58.415158+0000","last_undegraded":"2024-09-16T04:47:58.415158+0000","last_fullsized":"2024-09-16T04:47:58.415158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423488+0000","last_change":"2024-09-16T04:47:56.416639+0000","last_active":"2024-09-16T04:47:59.423488+0000","last_peered":"2024-09-16T04:47:59.423488+0000","last_clean":"2024-09-16T04:47:59.423488+0000","last_became_active":"2024-09-16T04:47:56.416184+0000","last_became_peered":"2024-09-16T04:47:56.416184+0000","last_unstale":"2024-09-16T04:47:59.423488+0000","last_undegraded":"2024-09-16T04:47:59.423488+0000","last_fullsized":"2024-09-16T04:47:59.423488+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:59.275401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423387+0000","last_change":"2024-09-16T04:47:56.416550+0000","last_active":"2024-09-16T04:47:59.423387+0000","last_peered":"2024-09-16T04:47:59.423387+0000","last_clean":"2024-09-16T04:47:59.423387+0000","last_became_active":"2024-09-16T04:47:56.416077+0000","last_became_peered":"2024-09-16T04:47:56.416077+0000","last_unstale":"2024-09-16T04:47:59.423387+0000","last_undegraded":"2024-09-16T04:47:59.423387+0000","last_fullsized":"2024-09-16T04:47:59.423387+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:14.550234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.412604+0000","last_change":"2024-09-16T04:47:58.412604+0000","last_active":"2024-09-16T04:47:58.412604+0000","last_peered":"2024-09-16T04:47:58.315475+0000","last_clean":"2024-09-16T04:47:58.315475+0000","last_became_active":"2024-09-16T04:47:56.401094+0000","last_became_peered":"2024-09-16T04:47:56.401094+0000","last_unstale":"2024-09-16T04:47:58.412604+0000","last_undegraded":"2024-09-16T04:47:58.412604+0000","last_fullsized":"2024-09-16T04:47:58.412604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.422693+0000","last_change":"2024-09-16T04:47:46.342776+0000","last_active":"2024-09-16T04:47:59.422693+0000","last_peered":"2024-09-16T04:47:59.422693+0000","last_clean":"2024-09-16T04:47:59.422693+0000","last_became_active":"2024-09-16T04:47:44.339613+0000","last_became_peered":"2024-09-16T04:47:44.339613+0000","last_unstale":"2024-09-16T04:47:59.422693+0000","last_undegraded":"2024-09-16T04:47:59.422693+0000","last_fullsized":"2024-09-16T04:47:59.422693+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:25.523051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317349+0000","last_change":"2024-09-16T04:47:58.413217+0000","last_active":"2024-09-16T04:47:58.413217+0000","last_peered":"2024-09-16T04:47:57.396117+0000","last_clean":"2024-09-16T04:47:57.396117+0000","last_became_active":"2024-09-16T04:47:56.413251+0000","last_became_peered":"2024-09-16T04:47:56.413251+0000","last_unstale":"2024-09-16T04:47:59.317349+0000","last_undegraded":"2024-09-16T04:47:59.317349+0000","last_fullsized":"2024-09-16T04:47:59.317349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-16T04:47:59.426984+0000","last_change":"2024-09-16T04:47:58.415328+0000","last_active":"2024-09-16T04:47:43.303429+0000","last_peered":"2024-09-16T04:47:43.303429+0000","last_clean":"2024-09-16T04:47:43.303429+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.426984+0000","last_undegraded":"2024-09-16T04:47:59.426984+0000","last_fullsized":"2024-09-16T04:47:59.426984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.427452+0000","last_change":"2024-09-16T04:47:56.406017+0000","last_active":"2024-09-16T04:47:59.427452+0000","last_peered":"2024-09-16T04:47:59.427452+0000","last_clean":"2024-09-16T04:47:59.427452+0000","last_became_active":"2024-09-16T04:47:56.405573+0000","last_became_peered":"2024-09-16T04:47:56.405573+0000","last_unstale":"2024-09-16T04:47:59.427452+0000","last_undegraded":"2024-09-16T04:47:59.427452+0000","last_fullsized":"2024-09-16T04:47:59.427452+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:31.014311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":33,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.427561+0000","last_change":"2024-09-16T04:47:46.341519+0000","last_active":"2024-09-16T04:47:59.427561+0000","last_peered":"2024-09-16T04:47:59.427561+0000","last_clean":"2024-09-16T04:47:59.427561+0000","last_became_active":"2024-09-16T04:47:44.337671+0000","last_became_peered":"2024-09-16T04:47:44.337671+0000","last_unstale":"2024-09-16T04:47:59.427561+0000","last_undegraded":"2024-09-16T04:47:59.427561+0000","last_fullsized":"2024-09-16T04:47:59.427561+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.766406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317288+0000","last_change":"2024-09-16T04:47:58.416259+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.317288+0000","last_undegraded":"2024-09-16T04:47:59.317288+0000","last_fullsized":"2024-09-16T04:47:59.317288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.414973+0000","last_change":"2024-09-16T04:47:58.414973+0000","last_active":"2024-09-16T04:47:58.414973+0000","last_peered":"2024-09-16T04:47:57.395966+0000","last_clean":"2024-09-16T04:47:57.395966+0000","last_became_active":"2024-09-16T04:47:44.337472+0000","last_became_peered":"2024-09-16T04:47:44.337472+0000","last_unstale":"2024-09-16T04:47:58.414973+0000","last_undegraded":"2024-09-16T04:47:58.414973+0000","last_fullsized":"2024-09-16T04:47:58.414973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317242+0000","last_change":"2024-09-16T04:47:58.412562+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.317242+0000","last_undegraded":"2024-09-16T04:47:59.317242+0000","last_fullsized":"2024-09-16T04:47:59.317242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423507+0000","last_change":"2024-09-16T04:47:56.412192+0000","last_active":"2024-09-16T04:47:59.423507+0000","last_peered":"2024-09-16T04:47:59.423507+0000","last_clean":"2024-09-16T04:47:59.423507+0000","last_became_active":"2024-09-16T04:47:56.411398+0000","last_became_peered":"2024-09-16T04:47:56.411398+0000","last_unstale":"2024-09-16T04:47:59.423507+0000","last_undegraded":"2024-09-16T04:47:59.423507+0000","last_fullsized":"2024-09-16T04:47:59.423507+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:58.421611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423619+0000","last_change":"2024-09-16T04:47:46.343554+0000","last_active":"2024-09-16T04:47:59.423619+0000","last_peered":"2024-09-16T04:47:59.423619+0000","last_clean":"2024-09-16T04:47:59.423619+0000","last_became_active":"2024-09-16T04:47:44.336016+0000","last_became_peered":"2024-09-16T04:47:44.336016+0000","last_unstale":"2024-09-16T04:47:59.423619+0000","last_undegraded":"2024-09-16T04:47:59.423619+0000","last_fullsized":"2024-09-16T04:47:59.423619+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:01.791016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423883+0000","last_change":"2024-09-16T04:47:56.416361+0000","last_active":"2024-09-16T04:47:59.423883+0000","last_peered":"2024-09-16T04:47:59.423883+0000","last_clean":"2024-09-16T04:47:59.423883+0000","last_became_active":"2024-09-16T04:47:56.409348+0000","last_became_peered":"2024-09-16T04:47:56.409348+0000","last_unstale":"2024-09-16T04:47:59.423883+0000","last_undegraded":"2024-09-16T04:47:59.423883+0000","last_fullsized":"2024-09-16T04:47:59.423883+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:07.297529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-16T04:47:59.429668+0000","last_change":"2024-09-16T04:47:58.418094+0000","last_active":"2024-09-16T04:47:43.303429+0000","last_peered":"2024-09-16T04:47:43.303429+0000","last_clean":"2024-09-16T04:47:43.303429+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.429668+0000","last_undegraded":"2024-09-16T04:47:59.429668+0000","last_fullsized":"2024-09-16T04:47:59.429668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.424911+0000","last_change":"2024-09-16T04:47:56.410174+0000","last_active":"2024-09-16T04:47:59.424911+0000","last_peered":"2024-09-16T04:47:59.424911+0000","last_clean":"2024-09-16T04:47:59.424911+0000","last_became_active":"2024-09-16T04:47:56.409473+0000","last_became_peered":"2024-09-16T04:47:56.409473+0000","last_unstale":"2024-09-16T04:47:59.424911+0000","last_undegraded":"2024-09-16T04:47:59.424911+0000","last_fullsized":"2024-09-16T04:47:59.424911+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:09.086995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.424116+0000","last_change":"2024-09-16T04:47:46.342646+0000","last_active":"2024-09-16T04:47:59.424116+0000","last_peered":"2024-09-16T04:47:59.424116+0000","last_clean":"2024-09-16T04:47:59.424116+0000","last_became_active":"2024-09-16T04:47:44.336027+0000","last_became_peered":"2024-09-16T04:47:44.336027+0000","last_unstale":"2024-09-16T04:47:59.424116+0000","last_undegraded":"2024-09-16T04:47:59.424116+0000","last_fullsized":"2024-09-16T04:47:59.424116+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:33.652853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":29,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.415940+0000","last_change":"2024-09-16T04:47:58.415940+0000","last_active":"2024-09-16T04:47:58.415940+0000","last_peered":"2024-09-16T04:47:57.396629+0000","last_clean":"2024-09-16T04:47:57.396629+0000","last_became_active":"2024-09-16T04:47:44.336686+0000","last_became_peered":"2024-09-16T04:47:44.336686+0000","last_unstale":"2024-09-16T04:47:58.415940+0000","last_undegraded":"2024-09-16T04:47:58.415940+0000","last_fullsized":"2024-09-16T04:47:58.415940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423505+0000","last_change":"2024-09-16T04:47:56.416536+0000","last_active":"2024-09-16T04:47:59.423505+0000","last_peered":"2024-09-16T04:47:59.423505+0000","last_clean":"2024-09-16T04:47:59.423505+0000","last_became_active":"2024-09-16T04:47:56.416197+0000","last_became_peered":"2024-09-16T04:47:56.416197+0000","last_unstale":"2024-09-16T04:47:59.423505+0000","last_undegraded":"2024-09-16T04:47:59.423505+0000","last_fullsized":"2024-09-16T04:47:59.423505+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:41.783205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":43108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":167936,"data_stored":43108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:48:02.020 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:48:02.317 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:02.318 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:48:02.335 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-16T04:48:02.250744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54392,"kb_used_data":328,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689288,"statfs":{"total":193273528320,"available":193217830912,"internally_reserved":0,"allocated":335872,"data_stored":86216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3188,"internal_metadata":55309196},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.857742"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.429370+0000","last_change":"2024-09-16T04:47:56.406884+0000","last_active":"2024-09-16T04:47:59.429370+0000","last_peered":"2024-09-16T04:47:59.429370+0000","last_clean":"2024-09-16T04:47:59.429370+0000","last_became_active":"2024-09-16T04:47:56.405974+0000","last_became_peered":"2024-09-16T04:47:56.405974+0000","last_unstale":"2024-09-16T04:47:59.429370+0000","last_undegraded":"2024-09-16T04:47:59.429370+0000","last_fullsized":"2024-09-16T04:47:59.429370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:04.264841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-16T04:47:59.430410+0000","last_change":"2024-09-16T04:47:58.417096+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.430410+0000","last_undegraded":"2024-09-16T04:47:59.430410+0000","last_fullsized":"2024-09-16T04:47:59.430410+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317395+0000","last_change":"2024-09-16T04:47:58.413540+0000","last_active":"2024-09-16T04:47:58.413540+0000","last_peered":"2024-09-16T04:47:57.396939+0000","last_clean":"2024-09-16T04:47:57.396939+0000","last_became_active":"2024-09-16T04:47:56.410391+0000","last_became_peered":"2024-09-16T04:47:56.410391+0000","last_unstale":"2024-09-16T04:47:59.317395+0000","last_undegraded":"2024-09-16T04:47:59.317395+0000","last_fullsized":"2024-09-16T04:47:59.317395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-16T04:47:59.429521+0000","last_change":"2024-09-16T04:47:58.416847+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.429521+0000","last_undegraded":"2024-09-16T04:47:59.429521+0000","last_fullsized":"2024-09-16T04:47:59.429521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.415158+0000","last_change":"2024-09-16T04:47:58.415158+0000","last_active":"2024-09-16T04:47:58.415158+0000","last_peered":"2024-09-16T04:47:58.315565+0000","last_clean":"2024-09-16T04:47:58.315565+0000","last_became_active":"2024-09-16T04:47:56.401138+0000","last_became_peered":"2024-09-16T04:47:56.401138+0000","last_unstale":"2024-09-16T04:47:58.415158+0000","last_undegraded":"2024-09-16T04:47:58.415158+0000","last_fullsized":"2024-09-16T04:47:58.415158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423488+0000","last_change":"2024-09-16T04:47:56.416639+0000","last_active":"2024-09-16T04:47:59.423488+0000","last_peered":"2024-09-16T04:47:59.423488+0000","last_clean":"2024-09-16T04:47:59.423488+0000","last_became_active":"2024-09-16T04:47:56.416184+0000","last_became_peered":"2024-09-16T04:47:56.416184+0000","last_unstale":"2024-09-16T04:47:59.423488+0000","last_undegraded":"2024-09-16T04:47:59.423488+0000","last_fullsized":"2024-09-16T04:47:59.423488+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:59.275401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423387+0000","last_change":"2024-09-16T04:47:56.416550+0000","last_active":"2024-09-16T04:47:59.423387+0000","last_peered":"2024-09-16T04:47:59.423387+0000","last_clean":"2024-09-16T04:47:59.423387+0000","last_became_active":"2024-09-16T04:47:56.416077+0000","last_became_peered":"2024-09-16T04:47:56.416077+0000","last_unstale":"2024-09-16T04:47:59.423387+0000","last_undegraded":"2024-09-16T04:47:59.423387+0000","last_fullsized":"2024-09-16T04:47:59.423387+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:14.550234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.412604+0000","last_change":"2024-09-16T04:47:58.412604+0000","last_active":"2024-09-16T04:47:58.412604+0000","last_peered":"2024-09-16T04:47:58.315475+0000","last_clean":"2024-09-16T04:47:58.315475+0000","last_became_active":"2024-09-16T04:47:56.401094+0000","last_became_peered":"2024-09-16T04:47:56.401094+0000","last_unstale":"2024-09-16T04:47:58.412604+0000","last_undegraded":"2024-09-16T04:47:58.412604+0000","last_fullsized":"2024-09-16T04:47:58.412604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.422693+0000","last_change":"2024-09-16T04:47:46.342776+0000","last_active":"2024-09-16T04:47:59.422693+0000","last_peered":"2024-09-16T04:47:59.422693+0000","last_clean":"2024-09-16T04:47:59.422693+0000","last_became_active":"2024-09-16T04:47:44.339613+0000","last_became_peered":"2024-09-16T04:47:44.339613+0000","last_unstale":"2024-09-16T04:47:59.422693+0000","last_undegraded":"2024-09-16T04:47:59.422693+0000","last_fullsized":"2024-09-16T04:47:59.422693+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:25.523051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317349+0000","last_change":"2024-09-16T04:47:58.413217+0000","last_active":"2024-09-16T04:47:58.413217+0000","last_peered":"2024-09-16T04:47:57.396117+0000","last_clean":"2024-09-16T04:47:57.396117+0000","last_became_active":"2024-09-16T04:47:56.413251+0000","last_became_peered":"2024-09-16T04:47:56.413251+0000","last_unstale":"2024-09-16T04:47:59.317349+0000","last_undegraded":"2024-09-16T04:47:59.317349+0000","last_fullsized":"2024-09-16T04:47:59.317349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-16T04:47:59.426984+0000","last_change":"2024-09-16T04:47:58.415328+0000","last_active":"2024-09-16T04:47:43.303429+0000","last_peered":"2024-09-16T04:47:43.303429+0000","last_clean":"2024-09-16T04:47:43.303429+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.426984+0000","last_undegraded":"2024-09-16T04:47:59.426984+0000","last_fullsized":"2024-09-16T04:47:59.426984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.427452+0000","last_change":"2024-09-16T04:47:56.406017+0000","last_active":"2024-09-16T04:47:59.427452+0000","last_peered":"2024-09-16T04:47:59.427452+0000","last_clean":"2024-09-16T04:47:59.427452+0000","last_became_active":"2024-09-16T04:47:56.405573+0000","last_became_peered":"2024-09-16T04:47:56.405573+0000","last_unstale":"2024-09-16T04:47:59.427452+0000","last_undegraded":"2024-09-16T04:47:59.427452+0000","last_fullsized":"2024-09-16T04:47:59.427452+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:31.014311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":33,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.427561+0000","last_change":"2024-09-16T04:47:46.341519+0000","last_active":"2024-09-16T04:47:59.427561+0000","last_peered":"2024-09-16T04:47:59.427561+0000","last_clean":"2024-09-16T04:47:59.427561+0000","last_became_active":"2024-09-16T04:47:44.337671+0000","last_became_peered":"2024-09-16T04:47:44.337671+0000","last_unstale":"2024-09-16T04:47:59.427561+0000","last_undegraded":"2024-09-16T04:47:59.427561+0000","last_fullsized":"2024-09-16T04:47:59.427561+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.766406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317288+0000","last_change":"2024-09-16T04:47:58.416259+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.317288+0000","last_undegraded":"2024-09-16T04:47:59.317288+0000","last_fullsized":"2024-09-16T04:47:59.317288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.414973+0000","last_change":"2024-09-16T04:47:58.414973+0000","last_active":"2024-09-16T04:47:58.414973+0000","last_peered":"2024-09-16T04:47:57.395966+0000","last_clean":"2024-09-16T04:47:57.395966+0000","last_became_active":"2024-09-16T04:47:44.337472+0000","last_became_peered":"2024-09-16T04:47:44.337472+0000","last_unstale":"2024-09-16T04:47:58.414973+0000","last_undegraded":"2024-09-16T04:47:58.414973+0000","last_fullsized":"2024-09-16T04:47:58.414973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317242+0000","last_change":"2024-09-16T04:47:58.412562+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.317242+0000","last_undegraded":"2024-09-16T04:47:59.317242+0000","last_fullsized":"2024-09-16T04:47:59.317242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423507+0000","last_change":"2024-09-16T04:47:56.412192+0000","last_active":"2024-09-16T04:47:59.423507+0000","last_peered":"2024-09-16T04:47:59.423507+0000","last_clean":"2024-09-16T04:47:59.423507+0000","last_became_active":"2024-09-16T04:47:56.411398+0000","last_became_peered":"2024-09-16T04:47:56.411398+0000","last_unstale":"2024-09-16T04:47:59.423507+0000","last_undegraded":"2024-09-16T04:47:59.423507+0000","last_fullsized":"2024-09-16T04:47:59.423507+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:58.421611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423619+0000","last_change":"2024-09-16T04:47:46.343554+0000","last_active":"2024-09-16T04:47:59.423619+0000","last_peered":"2024-09-16T04:47:59.423619+0000","last_clean":"2024-09-16T04:47:59.423619+0000","last_became_active":"2024-09-16T04:47:44.336016+0000","last_became_peered":"2024-09-16T04:47:44.336016+0000","last_unstale":"2024-09-16T04:47:59.423619+0000","last_undegraded":"2024-09-16T04:47:59.423619+0000","last_fullsized":"2024-09-16T04:47:59.423619+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:01.791016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423883+0000","last_change":"2024-09-16T04:47:56.416361+0000","last_active":"2024-09-16T04:47:59.423883+0000","last_peered":"2024-09-16T04:47:59.423883+0000","last_clean":"2024-09-16T04:47:59.423883+0000","last_became_active":"2024-09-16T04:47:56.409348+0000","last_became_peered":"2024-09-16T04:47:56.409348+0000","last_unstale":"2024-09-16T04:47:59.423883+0000","last_undegraded":"2024-09-16T04:47:59.423883+0000","last_fullsized":"2024-09-16T04:47:59.423883+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:07.297529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-16T04:47:59.429668+0000","last_change":"2024-09-16T04:47:58.418094+0000","last_active":"2024-09-16T04:47:43.303429+0000","last_peered":"2024-09-16T04:47:43.303429+0000","last_clean":"2024-09-16T04:47:43.303429+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.429668+0000","last_undegraded":"2024-09-16T04:47:59.429668+0000","last_fullsized":"2024-09-16T04:47:59.429668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.424911+0000","last_change":"2024-09-16T04:47:56.410174+0000","last_active":"2024-09-16T04:47:59.424911+0000","last_peered":"2024-09-16T04:47:59.424911+0000","last_clean":"2024-09-16T04:47:59.424911+0000","last_became_active":"2024-09-16T04:47:56.409473+0000","last_became_peered":"2024-09-16T04:47:56.409473+0000","last_unstale":"2024-09-16T04:47:59.424911+0000","last_undegraded":"2024-09-16T04:47:59.424911+0000","last_fullsized":"2024-09-16T04:47:59.424911+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:09.086995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.424116+0000","last_change":"2024-09-16T04:47:46.342646+0000","last_active":"2024-09-16T04:47:59.424116+0000","last_peered":"2024-09-16T04:47:59.424116+0000","last_clean":"2024-09-16T04:47:59.424116+0000","last_became_active":"2024-09-16T04:47:44.336027+0000","last_became_peered":"2024-09-16T04:47:44.336027+0000","last_unstale":"2024-09-16T04:47:59.424116+0000","last_undegraded":"2024-09-16T04:47:59.424116+0000","last_fullsized":"2024-09-16T04:47:59.424116+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:33.652853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":29,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.415940+0000","last_change":"2024-09-16T04:47:58.415940+0000","last_active":"2024-09-16T04:47:58.415940+0000","last_peered":"2024-09-16T04:47:57.396629+0000","last_clean":"2024-09-16T04:47:57.396629+0000","last_became_active":"2024-09-16T04:47:44.336686+0000","last_became_peered":"2024-09-16T04:47:44.336686+0000","last_unstale":"2024-09-16T04:47:58.415940+0000","last_undegraded":"2024-09-16T04:47:58.415940+0000","last_fullsized":"2024-09-16T04:47:58.415940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423505+0000","last_change":"2024-09-16T04:47:56.416536+0000","last_active":"2024-09-16T04:47:59.423505+0000","last_peered":"2024-09-16T04:47:59.423505+0000","last_clean":"2024-09-16T04:47:59.423505+0000","last_became_active":"2024-09-16T04:47:56.416197+0000","last_became_peered":"2024-09-16T04:47:56.416197+0000","last_unstale":"2024-09-16T04:47:59.423505+0000","last_undegraded":"2024-09-16T04:47:59.423505+0000","last_fullsized":"2024-09-16T04:47:59.423505+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:41.783205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":43108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":167936,"data_stored":43108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:48:05.340 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:48:05.650 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:48:05.668 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-16T04:48:04.251192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54408,"kb_used_data":348,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689272,"statfs":{"total":193273528320,"available":193217814528,"internally_reserved":0,"allocated":356352,"data_stored":92325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3188,"internal_metadata":55309196},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001536"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.428392+0000","last_change":"2024-09-16T04:47:56.406884+0000","last_active":"2024-09-16T04:48:00.428392+0000","last_peered":"2024-09-16T04:48:00.428392+0000","last_clean":"2024-09-16T04:48:00.428392+0000","last_became_active":"2024-09-16T04:47:56.405974+0000","last_became_peered":"2024-09-16T04:47:56.405974+0000","last_unstale":"2024-09-16T04:48:00.428392+0000","last_undegraded":"2024-09-16T04:48:00.428392+0000","last_fullsized":"2024-09-16T04:48:00.428392+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:04.264841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.445909+0000","last_change":"2024-09-16T04:48:00.445909+0000","last_active":"2024-09-16T04:48:00.445909+0000","last_peered":"2024-09-16T04:48:00.445909+0000","last_clean":"2024-09-16T04:48:00.445909+0000","last_became_active":"2024-09-16T04:48:00.445077+0000","last_became_peered":"2024-09-16T04:48:00.445077+0000","last_unstale":"2024-09-16T04:48:00.445909+0000","last_undegraded":"2024-09-16T04:48:00.445909+0000","last_fullsized":"2024-09-16T04:48:00.445909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:39:54.587297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317395+0000","last_change":"2024-09-16T04:47:58.413540+0000","last_active":"2024-09-16T04:47:58.413540+0000","last_peered":"2024-09-16T04:47:57.396939+0000","last_clean":"2024-09-16T04:47:57.396939+0000","last_became_active":"2024-09-16T04:47:56.410391+0000","last_became_peered":"2024-09-16T04:47:56.410391+0000","last_unstale":"2024-09-16T04:47:59.317395+0000","last_undegraded":"2024-09-16T04:47:59.317395+0000","last_fullsized":"2024-09-16T04:47:59.317395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.440996+0000","last_change":"2024-09-16T04:48:00.440996+0000","last_active":"2024-09-16T04:48:00.440996+0000","last_peered":"2024-09-16T04:48:00.440996+0000","last_clean":"2024-09-16T04:48:00.440996+0000","last_became_active":"2024-09-16T04:48:00.440316+0000","last_became_peered":"2024-09-16T04:48:00.440316+0000","last_unstale":"2024-09-16T04:48:00.440996+0000","last_undegraded":"2024-09-16T04:48:00.440996+0000","last_fullsized":"2024-09-16T04:48:00.440996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:03.845824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.447038+0000","last_change":"2024-09-16T04:48:00.447038+0000","last_active":"2024-09-16T04:48:00.447038+0000","last_peered":"2024-09-16T04:48:00.447038+0000","last_clean":"2024-09-16T04:48:00.447038+0000","last_became_active":"2024-09-16T04:48:00.445865+0000","last_became_peered":"2024-09-16T04:48:00.445865+0000","last_unstale":"2024-09-16T04:48:00.447038+0000","last_undegraded":"2024-09-16T04:48:00.447038+0000","last_fullsized":"2024-09-16T04:48:00.447038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:22.733373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423488+0000","last_change":"2024-09-16T04:47:56.416639+0000","last_active":"2024-09-16T04:47:59.423488+0000","last_peered":"2024-09-16T04:47:59.423488+0000","last_clean":"2024-09-16T04:47:59.423488+0000","last_became_active":"2024-09-16T04:47:56.416184+0000","last_became_peered":"2024-09-16T04:47:56.416184+0000","last_unstale":"2024-09-16T04:47:59.423488+0000","last_undegraded":"2024-09-16T04:47:59.423488+0000","last_fullsized":"2024-09-16T04:47:59.423488+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:59.275401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423387+0000","last_change":"2024-09-16T04:47:56.416550+0000","last_active":"2024-09-16T04:47:59.423387+0000","last_peered":"2024-09-16T04:47:59.423387+0000","last_clean":"2024-09-16T04:47:59.423387+0000","last_became_active":"2024-09-16T04:47:56.416077+0000","last_became_peered":"2024-09-16T04:47:56.416077+0000","last_unstale":"2024-09-16T04:47:59.423387+0000","last_undegraded":"2024-09-16T04:47:59.423387+0000","last_fullsized":"2024-09-16T04:47:59.423387+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:14.550234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.440148+0000","last_change":"2024-09-16T04:48:00.440148+0000","last_active":"2024-09-16T04:48:00.440148+0000","last_peered":"2024-09-16T04:48:00.440148+0000","last_clean":"2024-09-16T04:48:00.440148+0000","last_became_active":"2024-09-16T04:48:00.439410+0000","last_became_peered":"2024-09-16T04:48:00.439410+0000","last_unstale":"2024-09-16T04:48:00.440148+0000","last_undegraded":"2024-09-16T04:48:00.440148+0000","last_fullsized":"2024-09-16T04:48:00.440148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:46.837998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.422693+0000","last_change":"2024-09-16T04:47:46.342776+0000","last_active":"2024-09-16T04:47:59.422693+0000","last_peered":"2024-09-16T04:47:59.422693+0000","last_clean":"2024-09-16T04:47:59.422693+0000","last_became_active":"2024-09-16T04:47:44.339613+0000","last_became_peered":"2024-09-16T04:47:44.339613+0000","last_unstale":"2024-09-16T04:47:59.422693+0000","last_undegraded":"2024-09-16T04:47:59.422693+0000","last_fullsized":"2024-09-16T04:47:59.422693+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:25.523051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317349+0000","last_change":"2024-09-16T04:47:58.413217+0000","last_active":"2024-09-16T04:47:58.413217+0000","last_peered":"2024-09-16T04:47:57.396117+0000","last_clean":"2024-09-16T04:47:57.396117+0000","last_became_active":"2024-09-16T04:47:56.413251+0000","last_became_peered":"2024-09-16T04:47:56.413251+0000","last_unstale":"2024-09-16T04:47:59.317349+0000","last_undegraded":"2024-09-16T04:47:59.317349+0000","last_fullsized":"2024-09-16T04:47:59.317349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.440720+0000","last_change":"2024-09-16T04:48:00.440720+0000","last_active":"2024-09-16T04:48:00.440720+0000","last_peered":"2024-09-16T04:48:00.440720+0000","last_clean":"2024-09-16T04:48:00.440720+0000","last_became_active":"2024-09-16T04:48:00.439873+0000","last_became_peered":"2024-09-16T04:48:00.439873+0000","last_unstale":"2024-09-16T04:48:00.440720+0000","last_undegraded":"2024-09-16T04:48:00.440720+0000","last_fullsized":"2024-09-16T04:48:00.440720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:25.996095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.428258+0000","last_change":"2024-09-16T04:47:56.406017+0000","last_active":"2024-09-16T04:48:00.428258+0000","last_peered":"2024-09-16T04:48:00.428258+0000","last_clean":"2024-09-16T04:48:00.428258+0000","last_became_active":"2024-09-16T04:47:56.405573+0000","last_became_peered":"2024-09-16T04:47:56.405573+0000","last_unstale":"2024-09-16T04:48:00.428258+0000","last_undegraded":"2024-09-16T04:48:00.428258+0000","last_fullsized":"2024-09-16T04:48:00.428258+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:31.014311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":35,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.428579+0000","last_change":"2024-09-16T04:47:46.341519+0000","last_active":"2024-09-16T04:48:00.428579+0000","last_peered":"2024-09-16T04:48:00.428579+0000","last_clean":"2024-09-16T04:48:00.428579+0000","last_became_active":"2024-09-16T04:47:44.337671+0000","last_became_peered":"2024-09-16T04:47:44.337671+0000","last_unstale":"2024-09-16T04:48:00.428579+0000","last_undegraded":"2024-09-16T04:48:00.428579+0000","last_fullsized":"2024-09-16T04:48:00.428579+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.766406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317288+0000","last_change":"2024-09-16T04:47:58.416259+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.317288+0000","last_undegraded":"2024-09-16T04:47:59.317288+0000","last_fullsized":"2024-09-16T04:47:59.317288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.414973+0000","last_change":"2024-09-16T04:47:58.414973+0000","last_active":"2024-09-16T04:47:58.414973+0000","last_peered":"2024-09-16T04:47:57.395966+0000","last_clean":"2024-09-16T04:47:57.395966+0000","last_became_active":"2024-09-16T04:47:44.337472+0000","last_became_peered":"2024-09-16T04:47:44.337472+0000","last_unstale":"2024-09-16T04:47:58.414973+0000","last_undegraded":"2024-09-16T04:47:58.414973+0000","last_fullsized":"2024-09-16T04:47:58.414973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317242+0000","last_change":"2024-09-16T04:47:58.412562+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.317242+0000","last_undegraded":"2024-09-16T04:47:59.317242+0000","last_fullsized":"2024-09-16T04:47:59.317242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423507+0000","last_change":"2024-09-16T04:47:56.412192+0000","last_active":"2024-09-16T04:47:59.423507+0000","last_peered":"2024-09-16T04:47:59.423507+0000","last_clean":"2024-09-16T04:47:59.423507+0000","last_became_active":"2024-09-16T04:47:56.411398+0000","last_became_peered":"2024-09-16T04:47:56.411398+0000","last_unstale":"2024-09-16T04:47:59.423507+0000","last_undegraded":"2024-09-16T04:47:59.423507+0000","last_fullsized":"2024-09-16T04:47:59.423507+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:58.421611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423619+0000","last_change":"2024-09-16T04:47:46.343554+0000","last_active":"2024-09-16T04:47:59.423619+0000","last_peered":"2024-09-16T04:47:59.423619+0000","last_clean":"2024-09-16T04:47:59.423619+0000","last_became_active":"2024-09-16T04:47:44.336016+0000","last_became_peered":"2024-09-16T04:47:44.336016+0000","last_unstale":"2024-09-16T04:47:59.423619+0000","last_undegraded":"2024-09-16T04:47:59.423619+0000","last_fullsized":"2024-09-16T04:47:59.423619+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:01.791016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423883+0000","last_change":"2024-09-16T04:47:56.416361+0000","last_active":"2024-09-16T04:47:59.423883+0000","last_peered":"2024-09-16T04:47:59.423883+0000","last_clean":"2024-09-16T04:47:59.423883+0000","last_became_active":"2024-09-16T04:47:56.409348+0000","last_became_peered":"2024-09-16T04:47:56.409348+0000","last_unstale":"2024-09-16T04:47:59.423883+0000","last_undegraded":"2024-09-16T04:47:59.423883+0000","last_fullsized":"2024-09-16T04:47:59.423883+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:07.297529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.440975+0000","last_change":"2024-09-16T04:48:00.440975+0000","last_active":"2024-09-16T04:48:00.440975+0000","last_peered":"2024-09-16T04:48:00.440975+0000","last_clean":"2024-09-16T04:48:00.440975+0000","last_became_active":"2024-09-16T04:48:00.440175+0000","last_became_peered":"2024-09-16T04:48:00.440175+0000","last_unstale":"2024-09-16T04:48:00.440975+0000","last_undegraded":"2024-09-16T04:48:00.440975+0000","last_fullsized":"2024-09-16T04:48:00.440975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:38.770329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.424911+0000","last_change":"2024-09-16T04:47:56.410174+0000","last_active":"2024-09-16T04:47:59.424911+0000","last_peered":"2024-09-16T04:47:59.424911+0000","last_clean":"2024-09-16T04:47:59.424911+0000","last_became_active":"2024-09-16T04:47:56.409473+0000","last_became_peered":"2024-09-16T04:47:56.409473+0000","last_unstale":"2024-09-16T04:47:59.424911+0000","last_undegraded":"2024-09-16T04:47:59.424911+0000","last_fullsized":"2024-09-16T04:47:59.424911+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:09.086995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.424116+0000","last_change":"2024-09-16T04:47:46.342646+0000","last_active":"2024-09-16T04:47:59.424116+0000","last_peered":"2024-09-16T04:47:59.424116+0000","last_clean":"2024-09-16T04:47:59.424116+0000","last_became_active":"2024-09-16T04:47:44.336027+0000","last_became_peered":"2024-09-16T04:47:44.336027+0000","last_unstale":"2024-09-16T04:47:59.424116+0000","last_undegraded":"2024-09-16T04:47:59.424116+0000","last_fullsized":"2024-09-16T04:47:59.424116+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:33.652853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":29,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.415940+0000","last_change":"2024-09-16T04:47:58.415940+0000","last_active":"2024-09-16T04:47:58.415940+0000","last_peered":"2024-09-16T04:47:57.396629+0000","last_clean":"2024-09-16T04:47:57.396629+0000","last_became_active":"2024-09-16T04:47:44.336686+0000","last_became_peered":"2024-09-16T04:47:44.336686+0000","last_unstale":"2024-09-16T04:47:58.415940+0000","last_undegraded":"2024-09-16T04:47:58.415940+0000","last_fullsized":"2024-09-16T04:47:58.415940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423505+0000","last_change":"2024-09-16T04:47:56.416536+0000","last_active":"2024-09-16T04:47:59.423505+0000","last_peered":"2024-09-16T04:47:59.423505+0000","last_clean":"2024-09-16T04:47:59.423505+0000","last_became_active":"2024-09-16T04:47:56.416197+0000","last_became_peered":"2024-09-16T04:47:56.416197+0000","last_unstale":"2024-09-16T04:47:59.423505+0000","last_undegraded":"2024-09-16T04:47:59.423505+0000","last_fullsized":"2024-09-16T04:47:59.423505+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:41.783205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":43108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738377,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":49217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:48:05.669 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:48:05.953 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:05.954 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:48:05.967 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-16T04:48:04.251192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54408,"kb_used_data":348,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689272,"statfs":{"total":193273528320,"available":193217814528,"internally_reserved":0,"allocated":356352,"data_stored":92325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3188,"internal_metadata":55309196},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001536"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.428392+0000","last_change":"2024-09-16T04:47:56.406884+0000","last_active":"2024-09-16T04:48:00.428392+0000","last_peered":"2024-09-16T04:48:00.428392+0000","last_clean":"2024-09-16T04:48:00.428392+0000","last_became_active":"2024-09-16T04:47:56.405974+0000","last_became_peered":"2024-09-16T04:47:56.405974+0000","last_unstale":"2024-09-16T04:48:00.428392+0000","last_undegraded":"2024-09-16T04:48:00.428392+0000","last_fullsized":"2024-09-16T04:48:00.428392+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:04.264841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.445909+0000","last_change":"2024-09-16T04:48:00.445909+0000","last_active":"2024-09-16T04:48:00.445909+0000","last_peered":"2024-09-16T04:48:00.445909+0000","last_clean":"2024-09-16T04:48:00.445909+0000","last_became_active":"2024-09-16T04:48:00.445077+0000","last_became_peered":"2024-09-16T04:48:00.445077+0000","last_unstale":"2024-09-16T04:48:00.445909+0000","last_undegraded":"2024-09-16T04:48:00.445909+0000","last_fullsized":"2024-09-16T04:48:00.445909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:39:54.587297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317395+0000","last_change":"2024-09-16T04:47:58.413540+0000","last_active":"2024-09-16T04:47:58.413540+0000","last_peered":"2024-09-16T04:47:57.396939+0000","last_clean":"2024-09-16T04:47:57.396939+0000","last_became_active":"2024-09-16T04:47:56.410391+0000","last_became_peered":"2024-09-16T04:47:56.410391+0000","last_unstale":"2024-09-16T04:47:59.317395+0000","last_undegraded":"2024-09-16T04:47:59.317395+0000","last_fullsized":"2024-09-16T04:47:59.317395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.440996+0000","last_change":"2024-09-16T04:48:00.440996+0000","last_active":"2024-09-16T04:48:00.440996+0000","last_peered":"2024-09-16T04:48:00.440996+0000","last_clean":"2024-09-16T04:48:00.440996+0000","last_became_active":"2024-09-16T04:48:00.440316+0000","last_became_peered":"2024-09-16T04:48:00.440316+0000","last_unstale":"2024-09-16T04:48:00.440996+0000","last_undegraded":"2024-09-16T04:48:00.440996+0000","last_fullsized":"2024-09-16T04:48:00.440996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:03.845824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.447038+0000","last_change":"2024-09-16T04:48:00.447038+0000","last_active":"2024-09-16T04:48:00.447038+0000","last_peered":"2024-09-16T04:48:00.447038+0000","last_clean":"2024-09-16T04:48:00.447038+0000","last_became_active":"2024-09-16T04:48:00.445865+0000","last_became_peered":"2024-09-16T04:48:00.445865+0000","last_unstale":"2024-09-16T04:48:00.447038+0000","last_undegraded":"2024-09-16T04:48:00.447038+0000","last_fullsized":"2024-09-16T04:48:00.447038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:22.733373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423488+0000","last_change":"2024-09-16T04:47:56.416639+0000","last_active":"2024-09-16T04:47:59.423488+0000","last_peered":"2024-09-16T04:47:59.423488+0000","last_clean":"2024-09-16T04:47:59.423488+0000","last_became_active":"2024-09-16T04:47:56.416184+0000","last_became_peered":"2024-09-16T04:47:56.416184+0000","last_unstale":"2024-09-16T04:47:59.423488+0000","last_undegraded":"2024-09-16T04:47:59.423488+0000","last_fullsized":"2024-09-16T04:47:59.423488+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:59.275401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423387+0000","last_change":"2024-09-16T04:47:56.416550+0000","last_active":"2024-09-16T04:47:59.423387+0000","last_peered":"2024-09-16T04:47:59.423387+0000","last_clean":"2024-09-16T04:47:59.423387+0000","last_became_active":"2024-09-16T04:47:56.416077+0000","last_became_peered":"2024-09-16T04:47:56.416077+0000","last_unstale":"2024-09-16T04:47:59.423387+0000","last_undegraded":"2024-09-16T04:47:59.423387+0000","last_fullsized":"2024-09-16T04:47:59.423387+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:14.550234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.440148+0000","last_change":"2024-09-16T04:48:00.440148+0000","last_active":"2024-09-16T04:48:00.440148+0000","last_peered":"2024-09-16T04:48:00.440148+0000","last_clean":"2024-09-16T04:48:00.440148+0000","last_became_active":"2024-09-16T04:48:00.439410+0000","last_became_peered":"2024-09-16T04:48:00.439410+0000","last_unstale":"2024-09-16T04:48:00.440148+0000","last_undegraded":"2024-09-16T04:48:00.440148+0000","last_fullsized":"2024-09-16T04:48:00.440148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:46.837998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.422693+0000","last_change":"2024-09-16T04:47:46.342776+0000","last_active":"2024-09-16T04:47:59.422693+0000","last_peered":"2024-09-16T04:47:59.422693+0000","last_clean":"2024-09-16T04:47:59.422693+0000","last_became_active":"2024-09-16T04:47:44.339613+0000","last_became_peered":"2024-09-16T04:47:44.339613+0000","last_unstale":"2024-09-16T04:47:59.422693+0000","last_undegraded":"2024-09-16T04:47:59.422693+0000","last_fullsized":"2024-09-16T04:47:59.422693+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:25.523051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317349+0000","last_change":"2024-09-16T04:47:58.413217+0000","last_active":"2024-09-16T04:47:58.413217+0000","last_peered":"2024-09-16T04:47:57.396117+0000","last_clean":"2024-09-16T04:47:57.396117+0000","last_became_active":"2024-09-16T04:47:56.413251+0000","last_became_peered":"2024-09-16T04:47:56.413251+0000","last_unstale":"2024-09-16T04:47:59.317349+0000","last_undegraded":"2024-09-16T04:47:59.317349+0000","last_fullsized":"2024-09-16T04:47:59.317349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.440720+0000","last_change":"2024-09-16T04:48:00.440720+0000","last_active":"2024-09-16T04:48:00.440720+0000","last_peered":"2024-09-16T04:48:00.440720+0000","last_clean":"2024-09-16T04:48:00.440720+0000","last_became_active":"2024-09-16T04:48:00.439873+0000","last_became_peered":"2024-09-16T04:48:00.439873+0000","last_unstale":"2024-09-16T04:48:00.440720+0000","last_undegraded":"2024-09-16T04:48:00.440720+0000","last_fullsized":"2024-09-16T04:48:00.440720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:25.996095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.428258+0000","last_change":"2024-09-16T04:47:56.406017+0000","last_active":"2024-09-16T04:48:00.428258+0000","last_peered":"2024-09-16T04:48:00.428258+0000","last_clean":"2024-09-16T04:48:00.428258+0000","last_became_active":"2024-09-16T04:47:56.405573+0000","last_became_peered":"2024-09-16T04:47:56.405573+0000","last_unstale":"2024-09-16T04:48:00.428258+0000","last_undegraded":"2024-09-16T04:48:00.428258+0000","last_fullsized":"2024-09-16T04:48:00.428258+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:31.014311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":35,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.428579+0000","last_change":"2024-09-16T04:47:46.341519+0000","last_active":"2024-09-16T04:48:00.428579+0000","last_peered":"2024-09-16T04:48:00.428579+0000","last_clean":"2024-09-16T04:48:00.428579+0000","last_became_active":"2024-09-16T04:47:44.337671+0000","last_became_peered":"2024-09-16T04:47:44.337671+0000","last_unstale":"2024-09-16T04:48:00.428579+0000","last_undegraded":"2024-09-16T04:48:00.428579+0000","last_fullsized":"2024-09-16T04:48:00.428579+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.766406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317288+0000","last_change":"2024-09-16T04:47:58.416259+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.317288+0000","last_undegraded":"2024-09-16T04:47:59.317288+0000","last_fullsized":"2024-09-16T04:47:59.317288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.414973+0000","last_change":"2024-09-16T04:47:58.414973+0000","last_active":"2024-09-16T04:47:58.414973+0000","last_peered":"2024-09-16T04:47:57.395966+0000","last_clean":"2024-09-16T04:47:57.395966+0000","last_became_active":"2024-09-16T04:47:44.337472+0000","last_became_peered":"2024-09-16T04:47:44.337472+0000","last_unstale":"2024-09-16T04:47:58.414973+0000","last_undegraded":"2024-09-16T04:47:58.414973+0000","last_fullsized":"2024-09-16T04:47:58.414973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:59.317242+0000","last_change":"2024-09-16T04:47:58.412562+0000","last_active":"2024-09-16T04:47:55.367805+0000","last_peered":"2024-09-16T04:47:55.367805+0000","last_clean":"2024-09-16T04:47:55.367805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T04:47:59.317242+0000","last_undegraded":"2024-09-16T04:47:59.317242+0000","last_fullsized":"2024-09-16T04:47:59.317242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423507+0000","last_change":"2024-09-16T04:47:56.412192+0000","last_active":"2024-09-16T04:47:59.423507+0000","last_peered":"2024-09-16T04:47:59.423507+0000","last_clean":"2024-09-16T04:47:59.423507+0000","last_became_active":"2024-09-16T04:47:56.411398+0000","last_became_peered":"2024-09-16T04:47:56.411398+0000","last_unstale":"2024-09-16T04:47:59.423507+0000","last_undegraded":"2024-09-16T04:47:59.423507+0000","last_fullsized":"2024-09-16T04:47:59.423507+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:58.421611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423619+0000","last_change":"2024-09-16T04:47:46.343554+0000","last_active":"2024-09-16T04:47:59.423619+0000","last_peered":"2024-09-16T04:47:59.423619+0000","last_clean":"2024-09-16T04:47:59.423619+0000","last_became_active":"2024-09-16T04:47:44.336016+0000","last_became_peered":"2024-09-16T04:47:44.336016+0000","last_unstale":"2024-09-16T04:47:59.423619+0000","last_undegraded":"2024-09-16T04:47:59.423619+0000","last_fullsized":"2024-09-16T04:47:59.423619+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:01.791016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423883+0000","last_change":"2024-09-16T04:47:56.416361+0000","last_active":"2024-09-16T04:47:59.423883+0000","last_peered":"2024-09-16T04:47:59.423883+0000","last_clean":"2024-09-16T04:47:59.423883+0000","last_became_active":"2024-09-16T04:47:56.409348+0000","last_became_peered":"2024-09-16T04:47:56.409348+0000","last_unstale":"2024-09-16T04:47:59.423883+0000","last_undegraded":"2024-09-16T04:47:59.423883+0000","last_fullsized":"2024-09-16T04:47:59.423883+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:07.297529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.440975+0000","last_change":"2024-09-16T04:48:00.440975+0000","last_active":"2024-09-16T04:48:00.440975+0000","last_peered":"2024-09-16T04:48:00.440975+0000","last_clean":"2024-09-16T04:48:00.440975+0000","last_became_active":"2024-09-16T04:48:00.440175+0000","last_became_peered":"2024-09-16T04:48:00.440175+0000","last_unstale":"2024-09-16T04:48:00.440975+0000","last_undegraded":"2024-09-16T04:48:00.440975+0000","last_fullsized":"2024-09-16T04:48:00.440975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:38.770329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.424911+0000","last_change":"2024-09-16T04:47:56.410174+0000","last_active":"2024-09-16T04:47:59.424911+0000","last_peered":"2024-09-16T04:47:59.424911+0000","last_clean":"2024-09-16T04:47:59.424911+0000","last_became_active":"2024-09-16T04:47:56.409473+0000","last_became_peered":"2024-09-16T04:47:56.409473+0000","last_unstale":"2024-09-16T04:47:59.424911+0000","last_undegraded":"2024-09-16T04:47:59.424911+0000","last_fullsized":"2024-09-16T04:47:59.424911+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:09.086995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.424116+0000","last_change":"2024-09-16T04:47:46.342646+0000","last_active":"2024-09-16T04:47:59.424116+0000","last_peered":"2024-09-16T04:47:59.424116+0000","last_clean":"2024-09-16T04:47:59.424116+0000","last_became_active":"2024-09-16T04:47:44.336027+0000","last_became_peered":"2024-09-16T04:47:44.336027+0000","last_unstale":"2024-09-16T04:47:59.424116+0000","last_undegraded":"2024-09-16T04:47:59.424116+0000","last_fullsized":"2024-09-16T04:47:59.424116+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:33.652853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":29,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-16T04:47:58.415940+0000","last_change":"2024-09-16T04:47:58.415940+0000","last_active":"2024-09-16T04:47:58.415940+0000","last_peered":"2024-09-16T04:47:57.396629+0000","last_clean":"2024-09-16T04:47:57.396629+0000","last_became_active":"2024-09-16T04:47:44.336686+0000","last_became_peered":"2024-09-16T04:47:44.336686+0000","last_unstale":"2024-09-16T04:47:58.415940+0000","last_undegraded":"2024-09-16T04:47:58.415940+0000","last_fullsized":"2024-09-16T04:47:58.415940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T04:47:59.423505+0000","last_change":"2024-09-16T04:47:56.416536+0000","last_active":"2024-09-16T04:47:59.423505+0000","last_peered":"2024-09-16T04:47:59.423505+0000","last_clean":"2024-09-16T04:47:59.423505+0000","last_became_active":"2024-09-16T04:47:56.416197+0000","last_became_peered":"2024-09-16T04:47:56.416197+0000","last_unstale":"2024-09-16T04:47:59.423505+0000","last_undegraded":"2024-09-16T04:47:59.423505+0000","last_fullsized":"2024-09-16T04:47:59.423505+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:41.783205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":43108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738377,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":49217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:48:08.970 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:48:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:09.276 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:48:09.292 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-09-16T04:48:08.252282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54436,"kb_used_data":368,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689244,"statfs":{"total":193273528320,"available":193217785856,"internally_reserved":0,"allocated":376832,"data_stored":98434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3188,"internal_metadata":55309196},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.832960"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.428392+0000","last_change":"2024-09-16T04:47:56.406884+0000","last_active":"2024-09-16T04:48:00.428392+0000","last_peered":"2024-09-16T04:48:00.428392+0000","last_clean":"2024-09-16T04:48:00.428392+0000","last_became_active":"2024-09-16T04:47:56.405974+0000","last_became_peered":"2024-09-16T04:47:56.405974+0000","last_unstale":"2024-09-16T04:48:00.428392+0000","last_undegraded":"2024-09-16T04:48:00.428392+0000","last_fullsized":"2024-09-16T04:48:00.428392+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:04.264841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.445909+0000","last_change":"2024-09-16T04:48:00.445909+0000","last_active":"2024-09-16T04:48:00.445909+0000","last_peered":"2024-09-16T04:48:00.445909+0000","last_clean":"2024-09-16T04:48:00.445909+0000","last_became_active":"2024-09-16T04:48:00.445077+0000","last_became_peered":"2024-09-16T04:48:00.445077+0000","last_unstale":"2024-09-16T04:48:00.445909+0000","last_undegraded":"2024-09-16T04:48:00.445909+0000","last_fullsized":"2024-09-16T04:48:00.445909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:39:54.587297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.437042+0000","last_change":"2024-09-16T04:48:00.437042+0000","last_active":"2024-09-16T04:48:00.437042+0000","last_peered":"2024-09-16T04:48:00.437042+0000","last_clean":"2024-09-16T04:48:00.437042+0000","last_became_active":"2024-09-16T04:48:00.436207+0000","last_became_peered":"2024-09-16T04:48:00.436207+0000","last_unstale":"2024-09-16T04:48:00.437042+0000","last_undegraded":"2024-09-16T04:48:00.437042+0000","last_fullsized":"2024-09-16T04:48:00.437042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:39.755280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.440996+0000","last_change":"2024-09-16T04:48:00.440996+0000","last_active":"2024-09-16T04:48:00.440996+0000","last_peered":"2024-09-16T04:48:00.440996+0000","last_clean":"2024-09-16T04:48:00.440996+0000","last_became_active":"2024-09-16T04:48:00.440316+0000","last_became_peered":"2024-09-16T04:48:00.440316+0000","last_unstale":"2024-09-16T04:48:00.440996+0000","last_undegraded":"2024-09-16T04:48:00.440996+0000","last_fullsized":"2024-09-16T04:48:00.440996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:36:03.845824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.447038+0000","last_change":"2024-09-16T04:48:00.447038+0000","last_active":"2024-09-16T04:48:00.447038+0000","last_peered":"2024-09-16T04:48:00.447038+0000","last_clean":"2024-09-16T04:48:00.447038+0000","last_became_active":"2024-09-16T04:48:00.445865+0000","last_became_peered":"2024-09-16T04:48:00.445865+0000","last_unstale":"2024-09-16T04:48:00.447038+0000","last_undegraded":"2024-09-16T04:48:00.447038+0000","last_fullsized":"2024-09-16T04:48:00.447038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:22.733373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.427362+0000","last_change":"2024-09-16T04:47:56.416639+0000","last_active":"2024-09-16T04:48:00.427362+0000","last_peered":"2024-09-16T04:48:00.427362+0000","last_clean":"2024-09-16T04:48:00.427362+0000","last_became_active":"2024-09-16T04:47:56.416184+0000","last_became_peered":"2024-09-16T04:47:56.416184+0000","last_unstale":"2024-09-16T04:48:00.427362+0000","last_undegraded":"2024-09-16T04:48:00.427362+0000","last_fullsized":"2024-09-16T04:48:00.427362+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:59.275401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.429045+0000","last_change":"2024-09-16T04:47:56.416550+0000","last_active":"2024-09-16T04:48:00.429045+0000","last_peered":"2024-09-16T04:48:00.429045+0000","last_clean":"2024-09-16T04:48:00.429045+0000","last_became_active":"2024-09-16T04:47:56.416077+0000","last_became_peered":"2024-09-16T04:47:56.416077+0000","last_unstale":"2024-09-16T04:48:00.429045+0000","last_undegraded":"2024-09-16T04:48:00.429045+0000","last_fullsized":"2024-09-16T04:48:00.429045+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:08:14.550234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.440148+0000","last_change":"2024-09-16T04:48:00.440148+0000","last_active":"2024-09-16T04:48:00.440148+0000","last_peered":"2024-09-16T04:48:00.440148+0000","last_clean":"2024-09-16T04:48:00.440148+0000","last_became_active":"2024-09-16T04:48:00.439410+0000","last_became_peered":"2024-09-16T04:48:00.439410+0000","last_unstale":"2024-09-16T04:48:00.440148+0000","last_undegraded":"2024-09-16T04:48:00.440148+0000","last_fullsized":"2024-09-16T04:48:00.440148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:46.837998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":33,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.427482+0000","last_change":"2024-09-16T04:47:46.342776+0000","last_active":"2024-09-16T04:48:00.427482+0000","last_peered":"2024-09-16T04:48:00.427482+0000","last_clean":"2024-09-16T04:48:00.427482+0000","last_became_active":"2024-09-16T04:47:44.339613+0000","last_became_peered":"2024-09-16T04:47:44.339613+0000","last_unstale":"2024-09-16T04:48:00.427482+0000","last_undegraded":"2024-09-16T04:48:00.427482+0000","last_fullsized":"2024-09-16T04:48:00.427482+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:16:25.523051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.442979+0000","last_change":"2024-09-16T04:48:00.442979+0000","last_active":"2024-09-16T04:48:00.442979+0000","last_peered":"2024-09-16T04:48:00.442979+0000","last_clean":"2024-09-16T04:48:00.442979+0000","last_became_active":"2024-09-16T04:48:00.442226+0000","last_became_peered":"2024-09-16T04:48:00.442226+0000","last_unstale":"2024-09-16T04:48:00.442979+0000","last_undegraded":"2024-09-16T04:48:00.442979+0000","last_fullsized":"2024-09-16T04:48:00.442979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:01.608507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.440720+0000","last_change":"2024-09-16T04:48:00.440720+0000","last_active":"2024-09-16T04:48:00.440720+0000","last_peered":"2024-09-16T04:48:00.440720+0000","last_clean":"2024-09-16T04:48:00.440720+0000","last_became_active":"2024-09-16T04:48:00.439873+0000","last_became_peered":"2024-09-16T04:48:00.439873+0000","last_unstale":"2024-09-16T04:48:00.440720+0000","last_undegraded":"2024-09-16T04:48:00.440720+0000","last_fullsized":"2024-09-16T04:48:00.440720+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:25.996095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.428258+0000","last_change":"2024-09-16T04:47:56.406017+0000","last_active":"2024-09-16T04:48:00.428258+0000","last_peered":"2024-09-16T04:48:00.428258+0000","last_clean":"2024-09-16T04:48:00.428258+0000","last_became_active":"2024-09-16T04:47:56.405573+0000","last_became_peered":"2024-09-16T04:47:56.405573+0000","last_unstale":"2024-09-16T04:48:00.428258+0000","last_undegraded":"2024-09-16T04:48:00.428258+0000","last_fullsized":"2024-09-16T04:48:00.428258+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:31.014311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":35,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.428579+0000","last_change":"2024-09-16T04:47:46.341519+0000","last_active":"2024-09-16T04:48:00.428579+0000","last_peered":"2024-09-16T04:48:00.428579+0000","last_clean":"2024-09-16T04:48:00.428579+0000","last_became_active":"2024-09-16T04:47:44.337671+0000","last_became_peered":"2024-09-16T04:47:44.337671+0000","last_unstale":"2024-09-16T04:48:00.428579+0000","last_undegraded":"2024-09-16T04:48:00.428579+0000","last_fullsized":"2024-09-16T04:48:00.428579+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.766406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.442976+0000","last_change":"2024-09-16T04:48:00.442976+0000","last_active":"2024-09-16T04:48:00.442976+0000","last_peered":"2024-09-16T04:48:00.442976+0000","last_clean":"2024-09-16T04:48:00.442976+0000","last_became_active":"2024-09-16T04:48:00.442104+0000","last_became_peered":"2024-09-16T04:48:00.442104+0000","last_unstale":"2024-09-16T04:48:00.442976+0000","last_undegraded":"2024-09-16T04:48:00.442976+0000","last_fullsized":"2024-09-16T04:48:00.442976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:37:50.820220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":42,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.451330+0000","last_change":"2024-09-16T04:48:00.451330+0000","last_active":"2024-09-16T04:48:00.451330+0000","last_peered":"2024-09-16T04:48:00.451330+0000","last_clean":"2024-09-16T04:48:00.451330+0000","last_became_active":"2024-09-16T04:48:00.443093+0000","last_became_peered":"2024-09-16T04:48:00.443093+0000","last_unstale":"2024-09-16T04:48:00.451330+0000","last_undegraded":"2024-09-16T04:48:00.451330+0000","last_fullsized":"2024-09-16T04:48:00.451330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:40.320298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.438598+0000","last_change":"2024-09-16T04:48:00.438598+0000","last_active":"2024-09-16T04:48:00.438598+0000","last_peered":"2024-09-16T04:48:00.438598+0000","last_clean":"2024-09-16T04:48:00.438598+0000","last_became_active":"2024-09-16T04:48:00.437814+0000","last_became_peered":"2024-09-16T04:48:00.437814+0000","last_unstale":"2024-09-16T04:48:00.438598+0000","last_undegraded":"2024-09-16T04:48:00.438598+0000","last_fullsized":"2024-09-16T04:48:00.438598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:08.632842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.429318+0000","last_change":"2024-09-16T04:47:56.412192+0000","last_active":"2024-09-16T04:48:00.429318+0000","last_peered":"2024-09-16T04:48:00.429318+0000","last_clean":"2024-09-16T04:48:00.429318+0000","last_became_active":"2024-09-16T04:47:56.411398+0000","last_became_peered":"2024-09-16T04:47:56.411398+0000","last_unstale":"2024-09-16T04:48:00.429318+0000","last_undegraded":"2024-09-16T04:48:00.429318+0000","last_fullsized":"2024-09-16T04:48:00.429318+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:58.421611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":33,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.432769+0000","last_change":"2024-09-16T04:47:46.343554+0000","last_active":"2024-09-16T04:48:00.432769+0000","last_peered":"2024-09-16T04:48:00.432769+0000","last_clean":"2024-09-16T04:48:00.432769+0000","last_became_active":"2024-09-16T04:47:44.336016+0000","last_became_peered":"2024-09-16T04:47:44.336016+0000","last_unstale":"2024-09-16T04:48:00.432769+0000","last_undegraded":"2024-09-16T04:48:00.432769+0000","last_fullsized":"2024-09-16T04:48:00.432769+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:29:01.791016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.427808+0000","last_change":"2024-09-16T04:47:56.416361+0000","last_active":"2024-09-16T04:48:00.427808+0000","last_peered":"2024-09-16T04:48:00.427808+0000","last_clean":"2024-09-16T04:48:00.427808+0000","last_became_active":"2024-09-16T04:47:56.409348+0000","last_became_peered":"2024-09-16T04:47:56.409348+0000","last_unstale":"2024-09-16T04:48:00.427808+0000","last_undegraded":"2024-09-16T04:48:00.427808+0000","last_fullsized":"2024-09-16T04:48:00.427808+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:52:07.297529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.440975+0000","last_change":"2024-09-16T04:48:00.440975+0000","last_active":"2024-09-16T04:48:00.440975+0000","last_peered":"2024-09-16T04:48:00.440975+0000","last_clean":"2024-09-16T04:48:00.440975+0000","last_became_active":"2024-09-16T04:48:00.440175+0000","last_became_peered":"2024-09-16T04:48:00.440175+0000","last_unstale":"2024-09-16T04:48:00.440975+0000","last_undegraded":"2024-09-16T04:48:00.440975+0000","last_fullsized":"2024-09-16T04:48:00.440975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:41:38.770329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.427899+0000","last_change":"2024-09-16T04:47:56.410174+0000","last_active":"2024-09-16T04:48:00.427899+0000","last_peered":"2024-09-16T04:48:00.427899+0000","last_clean":"2024-09-16T04:48:00.427899+0000","last_became_active":"2024-09-16T04:47:56.409473+0000","last_became_peered":"2024-09-16T04:47:56.409473+0000","last_unstale":"2024-09-16T04:48:00.427899+0000","last_undegraded":"2024-09-16T04:48:00.427899+0000","last_fullsized":"2024-09-16T04:48:00.427899+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:09.086995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":33,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.428045+0000","last_change":"2024-09-16T04:47:46.342646+0000","last_active":"2024-09-16T04:48:00.428045+0000","last_peered":"2024-09-16T04:48:00.428045+0000","last_clean":"2024-09-16T04:48:00.428045+0000","last_became_active":"2024-09-16T04:47:44.336027+0000","last_became_peered":"2024-09-16T04:47:44.336027+0000","last_unstale":"2024-09-16T04:48:00.428045+0000","last_undegraded":"2024-09-16T04:48:00.428045+0000","last_fullsized":"2024-09-16T04:48:00.428045+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:02:33.652853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.437643+0000","last_change":"2024-09-16T04:48:00.437643+0000","last_active":"2024-09-16T04:48:00.437643+0000","last_peered":"2024-09-16T04:48:00.437643+0000","last_clean":"2024-09-16T04:48:00.437643+0000","last_became_active":"2024-09-16T04:48:00.436457+0000","last_became_peered":"2024-09-16T04:48:00.436457+0000","last_unstale":"2024-09-16T04:48:00.437643+0000","last_undegraded":"2024-09-16T04:48:00.437643+0000","last_fullsized":"2024-09-16T04:48:00.437643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:38:26.274037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T04:48:00.432555+0000","last_change":"2024-09-16T04:47:56.416536+0000","last_active":"2024-09-16T04:48:00.432555+0000","last_peered":"2024-09-16T04:48:00.432555+0000","last_clean":"2024-09-16T04:48:00.432555+0000","last_became_active":"2024-09-16T04:47:56.416197+0000","last_became_peered":"2024-09-16T04:47:56.416197+0000","last_unstale":"2024-09-16T04:48:00.432555+0000","last_undegraded":"2024-09-16T04:48:00.432555+0000","last_fullsized":"2024-09-16T04:48:00.432555+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:41.783205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705673,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":188416,"data_stored":49217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738377,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":49217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:48:09.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-16T04:48:09.293 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-16T04:48:09.293 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-16T04:48:09.338 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-16T04:48:09.338 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-16T04:48:09.374 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_1 2024-09-16T04:48:09.374 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_1 2024-09-16T04:48:09.453 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-16T04:48:09.453 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-16T04:48:09.536 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-16T04:48:09.536 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-16T04:48:09.616 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_2 2024-09-16T04:48:09.616 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_2 2024-09-16T04:48:09.692 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-16T04:48:09.692 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-16T04:48:09.772 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-16T04:48:09.772 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-16T04:48:09.849 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_3 2024-09-16T04:48:09.849 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_3 2024-09-16T04:48:09.935 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-16T04:48:09.936 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-16T04:48:10.025 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-16T04:48:10.026 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-16T04:48:10.108 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_4 2024-09-16T04:48:10.108 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_4 2024-09-16T04:48:10.184 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-16T04:48:10.184 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-16T04:48:10.274 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-16T04:48:10.274 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-16T04:48:10.311 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_5 2024-09-16T04:48:10.311 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_5 2024-09-16T04:48:10.387 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-16T04:48:10.388 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-16T04:48:10.465 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-16T04:48:10.465 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-16T04:48:10.545 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_6 2024-09-16T04:48:10.546 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_6 2024-09-16T04:48:10.633 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-16T04:48:10.633 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-16T04:48:10.714 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-16T04:48:10.714 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-16T04:48:10.797 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_7 2024-09-16T04:48:10.797 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_7 2024-09-16T04:48:10.878 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-16T04:48:10.878 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-16T04:48:10.960 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-16T04:48:10.960 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-16T04:48:11.041 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_8 2024-09-16T04:48:11.041 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_8 2024-09-16T04:48:11.124 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-16T04:48:11.124 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-16T04:48:11.206 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-16T04:48:11.206 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-16T04:48:11.296 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_9 2024-09-16T04:48:11.296 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_9 2024-09-16T04:48:11.381 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 injectargs '--osd-recovery-delay-start 1000 --osd-min-pg-log-entries 100000000' 2024-09-16T04:48:11.572 INFO:teuthology.orchestra.run.smithi012.stdout:{} 2024-09-16T04:48:11.573 INFO:teuthology.orchestra.run.smithi012.stderr:osd_min_pg_log_entries = '' (not observed, change may require restart) osd_recovery_delay_start = '' 2024-09-16T04:48:11.588 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T04:48:11.588 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T04:48:11.675 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T04:48:11.675 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-16T04:48:11.986 INFO:tasks.ceph.mon.c.smithi012.stderr:2024-09-16T04:48:11.981+0000 7fbe3b864640 -1 mon.c@2(leader).osd e21 definitely_dead 0 2024-09-16T04:48:12.341 INFO:teuthology.orchestra.run.smithi012.stderr:marked down osd.0. 2024-09-16T04:48:12.361 INFO:tasks.util.rados:rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-16T04:48:12.362 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-16T04:48:13.357 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-16T04:48:13.357 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-16T04:48:13.403 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-16T04:48:13.403 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-16T04:48:13.441 INFO:tasks.util.rados:rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-16T04:48:13.441 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-16T04:48:13.524 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-16T04:48:13.524 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-16T04:48:13.604 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-16T04:48:13.604 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-16T04:48:13.689 INFO:tasks.util.rados:rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-16T04:48:13.689 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-16T04:48:13.770 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-16T04:48:13.771 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-16T04:48:13.857 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-16T04:48:13.857 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-16T04:48:13.942 INFO:tasks.util.rados:rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-16T04:48:13.942 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-16T04:48:14.023 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-16T04:48:14.024 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-16T04:48:14.101 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-16T04:48:14.101 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-16T04:48:14.187 INFO:tasks.util.rados:rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-16T04:48:14.187 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-16T04:48:14.273 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-16T04:48:14.274 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-16T04:48:14.357 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-16T04:48:14.358 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-16T04:48:14.438 INFO:tasks.util.rados:rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-16T04:48:14.438 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-16T04:48:14.517 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-16T04:48:14.517 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-16T04:48:14.605 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-16T04:48:14.605 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-16T04:48:14.687 INFO:tasks.util.rados:rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-16T04:48:14.687 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-16T04:48:14.772 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-16T04:48:14.772 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-16T04:48:14.851 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-16T04:48:14.851 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-16T04:48:14.929 INFO:tasks.util.rados:rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-16T04:48:14.929 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-16T04:48:15.010 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-16T04:48:15.010 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-16T04:48:15.091 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-16T04:48:15.091 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-16T04:48:15.179 INFO:tasks.util.rados:rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-16T04:48:15.179 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-16T04:48:15.217 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-16T04:48:15.217 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-16T04:48:15.310 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-16T04:48:15.310 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-16T04:48:15.350 INFO:tasks.rep_lost_unfound_delete:osd.0 command_args is foo 2024-09-16T04:48:15.350 INFO:tasks.rep_lost_unfound_delete:() 2024-09-16T04:48:15.350 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T04:48:15.350 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 --osd-recovery-delay-start 1000 2024-09-16T04:48:15.394 INFO:tasks.ceph.osd.0:Started 2024-09-16T04:48:15.395 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T04:48:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:no valid command found; 10 closest matches: 2024-09-16T04:48:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:0 2024-09-16T04:48:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:1 2024-09-16T04:48:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:2 2024-09-16T04:48:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:abort 2024-09-16T04:48:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:assert 2024-09-16T04:48:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:config diff 2024-09-16T04:48:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:config diff get 2024-09-16T04:48:15.514 INFO:teuthology.orchestra.run.smithi012.stderr:config get 2024-09-16T04:48:15.515 INFO:teuthology.orchestra.run.smithi012.stderr:config help [] 2024-09-16T04:48:15.515 INFO:teuthology.orchestra.run.smithi012.stderr:config set ... 2024-09-16T04:48:15.515 INFO:teuthology.orchestra.run.smithi012.stderr:admin_socket: invalid command 2024-09-16T04:48:15.516 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T04:48:15.517 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-16T04:48:17.411 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:17.405+0000 7f2e843827c0 -1 Falling back to public interface 2024-09-16T04:48:20.517 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T04:48:20.646 INFO:teuthology.orchestra.run.smithi012.stderr:no valid command found; 10 closest matches: 2024-09-16T04:48:20.646 INFO:teuthology.orchestra.run.smithi012.stderr:0 2024-09-16T04:48:20.646 INFO:teuthology.orchestra.run.smithi012.stderr:1 2024-09-16T04:48:20.646 INFO:teuthology.orchestra.run.smithi012.stderr:2 2024-09-16T04:48:20.647 INFO:teuthology.orchestra.run.smithi012.stderr:abort 2024-09-16T04:48:20.647 INFO:teuthology.orchestra.run.smithi012.stderr:assert 2024-09-16T04:48:20.647 INFO:teuthology.orchestra.run.smithi012.stderr:config diff 2024-09-16T04:48:20.647 INFO:teuthology.orchestra.run.smithi012.stderr:config diff get 2024-09-16T04:48:20.647 INFO:teuthology.orchestra.run.smithi012.stderr:config get 2024-09-16T04:48:20.647 INFO:teuthology.orchestra.run.smithi012.stderr:config help [] 2024-09-16T04:48:20.647 INFO:teuthology.orchestra.run.smithi012.stderr:config set ... 2024-09-16T04:48:20.647 INFO:teuthology.orchestra.run.smithi012.stderr:admin_socket: invalid command 2024-09-16T04:48:20.649 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T04:48:20.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-16T04:48:21.891 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:21.885+0000 7f2e843827c0 -1 osd.0 21 log_to_monitors true 2024-09-16T04:48:21.966 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:21.961+0000 7f2e7c157640 -1 osd.0 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T04:48:25.651 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T04:48:25.786 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:48:25.786 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-16T04:48:25.786 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-16T04:48:25.786 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:48:25.800 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-16T04:48:26.110 INFO:tasks.ceph.mon.c.smithi012.stderr:2024-09-16T04:48:26.105+0000 7fbe3b864640 -1 mon.c@2(leader).osd e26 definitely_dead 0 2024-09-16T04:48:26.110 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0 is already in. 2024-09-16T04:48:26.128 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.0 to be up 2024-09-16T04:48:26.129 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T04:48:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:26.423 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","created":"2024-09-16T04:47:27.160076+0000","modified":"2024-09-16T04:48:23.964984+0000","last_up_change":"2024-09-16T04:48:22.955575+0000","last_in_change":"2024-09-16T04:47:59.406388+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-16T04:47:42.612259+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-16T04:47:55.339654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7fa4b8a1-d218-47cb-adc4-469674c829fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":21,"up_from":25,"up_thru":25,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6800","nonce":2560862748}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6801","nonce":2560862748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6803","nonce":2560862748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6802","nonce":2560862748}]},"public_addr":"172.21.15.12:6800/2560862748","cluster_addr":"172.21.15.12:6801/2560862748","heartbeat_back_addr":"172.21.15.12:6803/2560862748","heartbeat_front_addr":"172.21.15.12:6802/2560862748","state":["exists","up"]},{"osd":1,"uuid":"7234f3ae-2cf3-4dd4-8c21-4f170976dbf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6808","nonce":1033795119}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6809","nonce":1033795119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6811","nonce":1033795119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6810","nonce":1033795119}]},"public_addr":"172.21.15.12:6808/1033795119","cluster_addr":"172.21.15.12:6809/1033795119","heartbeat_back_addr":"172.21.15.12:6811/1033795119","heartbeat_front_addr":"172.21.15.12:6810/1033795119","state":["exists","up"]},{"osd":2,"uuid":"919a351e-ecee-4e06-bff1-ae68c80d320f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6800","nonce":2135679025}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6801","nonce":2135679025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6803","nonce":2135679025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6802","nonce":2135679025}]},"public_addr":"172.21.15.12:6800/2135679025","cluster_addr":"172.21.15.12:6801/2135679025","heartbeat_back_addr":"172.21.15.12:6803/2135679025","heartbeat_front_addr":"172.21.15.12:6802/2135679025","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.109010+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:35.949029+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-16T04:47:36.023975+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T04:48:26.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.0 is up 2024-09-16T04:48:26.442 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T04:48:26.443 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T04:48:26.603 INFO:teuthology.orchestra.run.smithi012.stdout:107374182402 2024-09-16T04:48:26.603 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:48:26.606 INFO:teuthology.orchestra.run.smithi012.stdout:38654705678 2024-09-16T04:48:26.606 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T04:48:26.889 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-09-16T04:48:26.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182402 got 1 for osd.0 2024-09-16T04:48:26.943 INFO:teuthology.orchestra.run.smithi012.stdout:38654705676 2024-09-16T04:48:26.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705678 got 38654705676 for osd.1 2024-09-16T04:48:27.913 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:48:27.960 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T04:48:28.221 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-09-16T04:48:28.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182402 got 1 for osd.0 2024-09-16T04:48:28.252 INFO:teuthology.orchestra.run.smithi012.stdout:38654705677 2024-09-16T04:48:28.269 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705678 got 38654705677 for osd.1 2024-09-16T04:48:29.239 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:48:29.270 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T04:48:29.554 INFO:teuthology.orchestra.run.smithi012.stdout:107374182403 2024-09-16T04:48:29.572 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182402 got 107374182403 for osd.0 2024-09-16T04:48:29.572 DEBUG:teuthology.parallel:result is None 2024-09-16T04:48:29.586 INFO:teuthology.orchestra.run.smithi012.stdout:38654705678 2024-09-16T04:48:29.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705678 got 38654705678 for osd.1 2024-09-16T04:48:29.604 DEBUG:teuthology.parallel:result is None 2024-09-16T04:48:29.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-16T04:48:29.604 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:48:29.940 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:29.940 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:48:29.957 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2024-09-16T04:48:28.257804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55108,"kb_used_data":592,"kb_used_omap":5,"kb_used_meta":54458,"kb_avail":188688572,"statfs":{"total":193273528320,"available":193217097728,"internally_reserved":0,"allocated":606208,"data_stored":151614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6033,"internal_metadata":55765103},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.308242"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T04:48:23.995107+0000","last_change":"2024-09-16T04:48:23.995107+0000","last_active":"2024-09-16T04:48:23.995107+0000","last_peered":"2024-09-16T04:48:23.995107+0000","last_clean":"2024-09-16T04:48:23.995107+0000","last_became_active":"2024-09-16T04:48:23.994398+0000","last_became_peered":"2024-09-16T04:48:23.994398+0000","last_unstale":"2024-09-16T04:48:23.995107+0000","last_undegraded":"2024-09-16T04:48:23.995107+0000","last_fullsized":"2024-09-16T04:48:23.995107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:26.506492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":23,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:23.993104+0000","last_change":"2024-09-16T04:48:23.993575+0000","last_active":"2024-09-16T04:48:23.993104+0000","last_peered":"2024-09-16T04:48:23.993104+0000","last_clean":"2024-09-16T04:48:10.022495+0000","last_became_active":"2024-09-16T04:48:23.992838+0000","last_became_peered":"2024-09-16T04:48:23.992838+0000","last_unstale":"2024-09-16T04:48:23.993104+0000","last_undegraded":"2024-09-16T04:48:23.976957+0000","last_fullsized":"2024-09-16T04:48:23.993104+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":41,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:23.998737+0000","last_change":"2024-09-16T04:48:23.999193+0000","last_active":"2024-09-16T04:48:23.998737+0000","last_peered":"2024-09-16T04:48:23.998737+0000","last_clean":"2024-09-16T04:48:10.874965+0000","last_became_active":"2024-09-16T04:48:23.998510+0000","last_became_peered":"2024-09-16T04:48:23.998510+0000","last_unstale":"2024-09-16T04:48:23.998737+0000","last_undegraded":"2024-09-16T04:48:23.980214+0000","last_fullsized":"2024-09-16T04:48:23.998737+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T04:48:23.994021+0000","last_change":"2024-09-16T04:48:23.994021+0000","last_active":"2024-09-16T04:48:23.994021+0000","last_peered":"2024-09-16T04:48:23.994021+0000","last_clean":"2024-09-16T04:48:23.994021+0000","last_became_active":"2024-09-16T04:48:23.993201+0000","last_became_peered":"2024-09-16T04:48:23.993201+0000","last_unstale":"2024-09-16T04:48:23.994021+0000","last_undegraded":"2024-09-16T04:48:23.994021+0000","last_fullsized":"2024-09-16T04:48:23.994021+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:59.456741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":34,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:23.992450+0000","last_change":"2024-09-16T04:48:23.992939+0000","last_active":"2024-09-16T04:48:23.992450+0000","last_peered":"2024-09-16T04:48:23.992450+0000","last_clean":"2024-09-16T04:48:09.846075+0000","last_became_active":"2024-09-16T04:48:23.992190+0000","last_became_peered":"2024-09-16T04:48:23.992190+0000","last_unstale":"2024-09-16T04:48:23.992450+0000","last_undegraded":"2024-09-16T04:48:23.976787+0000","last_fullsized":"2024-09-16T04:48:23.992450+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":38,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-16T04:48:23.996257+0000","last_change":"2024-09-16T04:48:24.006905+0000","last_active":"2024-09-16T04:48:23.996257+0000","last_peered":"2024-09-16T04:48:23.996257+0000","last_clean":"2024-09-16T04:48:11.202938+0000","last_became_active":"2024-09-16T04:48:23.995653+0000","last_became_peered":"2024-09-16T04:48:23.995653+0000","last_unstale":"2024-09-16T04:48:23.996257+0000","last_undegraded":"2024-09-16T04:48:23.979453+0000","last_fullsized":"2024-09-16T04:48:23.996257+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":38,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:23.998261+0000","last_change":"2024-09-16T04:48:23.998718+0000","last_active":"2024-09-16T04:48:23.998261+0000","last_peered":"2024-09-16T04:48:23.998261+0000","last_clean":"2024-09-16T04:48:10.270497+0000","last_became_active":"2024-09-16T04:48:23.998107+0000","last_became_peered":"2024-09-16T04:48:23.998107+0000","last_unstale":"2024-09-16T04:48:23.998261+0000","last_undegraded":"2024-09-16T04:48:23.980435+0000","last_fullsized":"2024-09-16T04:48:23.998261+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":32,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-16T04:48:23.991398+0000","last_change":"2024-09-16T04:48:24.004232+0000","last_active":"2024-09-16T04:48:23.991398+0000","last_peered":"2024-09-16T04:48:23.991398+0000","last_clean":"2024-09-16T04:47:58.315475+0000","last_became_active":"2024-09-16T04:48:23.990860+0000","last_became_peered":"2024-09-16T04:48:23.990860+0000","last_unstale":"2024-09-16T04:48:23.991398+0000","last_undegraded":"2024-09-16T04:48:23.976335+0000","last_fullsized":"2024-09-16T04:48:23.991398+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T04:48:23.996285+0000","last_change":"2024-09-16T04:48:23.996285+0000","last_active":"2024-09-16T04:48:23.996285+0000","last_peered":"2024-09-16T04:48:23.996285+0000","last_clean":"2024-09-16T04:48:23.996285+0000","last_became_active":"2024-09-16T04:48:23.995430+0000","last_became_peered":"2024-09-16T04:48:23.995430+0000","last_unstale":"2024-09-16T04:48:23.996285+0000","last_undegraded":"2024-09-16T04:48:23.996285+0000","last_fullsized":"2024-09-16T04:48:23.996285+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:43:46.465967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":43,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:24.000815+0000","last_change":"2024-09-16T04:48:24.001191+0000","last_active":"2024-09-16T04:48:24.000815+0000","last_peered":"2024-09-16T04:48:24.000815+0000","last_clean":"2024-09-16T04:48:10.956703+0000","last_became_active":"2024-09-16T04:48:24.000556+0000","last_became_peered":"2024-09-16T04:48:24.000556+0000","last_unstale":"2024-09-16T04:48:24.000815+0000","last_undegraded":"2024-09-16T04:48:23.980482+0000","last_fullsized":"2024-09-16T04:48:24.000815+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T04:48:23.991650+0000","last_change":"2024-09-16T04:48:23.991650+0000","last_active":"2024-09-16T04:48:23.991650+0000","last_peered":"2024-09-16T04:48:23.991650+0000","last_clean":"2024-09-16T04:48:23.991650+0000","last_became_active":"2024-09-16T04:48:23.990748+0000","last_became_peered":"2024-09-16T04:48:23.990748+0000","last_unstale":"2024-09-16T04:48:23.991650+0000","last_undegraded":"2024-09-16T04:48:23.991650+0000","last_fullsized":"2024-09-16T04:48:23.991650+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:11:30.150761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":31,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:23.993249+0000","last_change":"2024-09-16T04:48:23.993709+0000","last_active":"2024-09-16T04:48:23.993249+0000","last_peered":"2024-09-16T04:48:23.993249+0000","last_clean":"2024-09-16T04:48:11.292504+0000","last_became_active":"2024-09-16T04:48:23.992991+0000","last_became_peered":"2024-09-16T04:48:23.992991+0000","last_unstale":"2024-09-16T04:48:23.993249+0000","last_undegraded":"2024-09-16T04:48:23.979041+0000","last_fullsized":"2024-09-16T04:48:23.993249+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":36,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T04:48:23.994097+0000","last_change":"2024-09-16T04:48:23.994097+0000","last_active":"2024-09-16T04:48:23.994097+0000","last_peered":"2024-09-16T04:48:23.994097+0000","last_clean":"2024-09-16T04:48:23.994097+0000","last_became_active":"2024-09-16T04:48:23.993223+0000","last_became_peered":"2024-09-16T04:48:23.993223+0000","last_unstale":"2024-09-16T04:48:23.994097+0000","last_undegraded":"2024-09-16T04:48:23.994097+0000","last_fullsized":"2024-09-16T04:48:23.994097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:17.733276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":31,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:23.995812+0000","last_change":"2024-09-16T04:48:23.996316+0000","last_active":"2024-09-16T04:48:23.995812+0000","last_peered":"2024-09-16T04:48:23.995812+0000","last_clean":"2024-09-16T04:48:09.768821+0000","last_became_active":"2024-09-16T04:48:23.995577+0000","last_became_peered":"2024-09-16T04:48:23.995577+0000","last_unstale":"2024-09-16T04:48:23.995812+0000","last_undegraded":"2024-09-16T04:48:23.979245+0000","last_fullsized":"2024-09-16T04:48:23.995812+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":59,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T04:48:24.001641+0000","last_change":"2024-09-16T04:48:24.001641+0000","last_active":"2024-09-16T04:48:24.001641+0000","last_peered":"2024-09-16T04:48:24.001641+0000","last_clean":"2024-09-16T04:48:24.001641+0000","last_became_active":"2024-09-16T04:48:24.000655+0000","last_became_peered":"2024-09-16T04:48:24.000655+0000","last_unstale":"2024-09-16T04:48:24.001641+0000","last_undegraded":"2024-09-16T04:48:24.001641+0000","last_fullsized":"2024-09-16T04:48:24.001641+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:49.485186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":32,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:23.997077+0000","last_change":"2024-09-16T04:48:23.997669+0000","last_active":"2024-09-16T04:48:23.997077+0000","last_peered":"2024-09-16T04:48:23.997077+0000","last_clean":"2024-09-16T04:48:10.384545+0000","last_became_active":"2024-09-16T04:48:23.996892+0000","last_became_peered":"2024-09-16T04:48:23.996892+0000","last_unstale":"2024-09-16T04:48:23.997077+0000","last_undegraded":"2024-09-16T04:48:23.979617+0000","last_fullsized":"2024-09-16T04:48:23.997077+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":34,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:24.001315+0000","last_change":"2024-09-16T04:48:24.002119+0000","last_active":"2024-09-16T04:48:24.001315+0000","last_peered":"2024-09-16T04:48:24.001315+0000","last_clean":"2024-09-16T04:48:00.429368+0000","last_became_active":"2024-09-16T04:48:24.001074+0000","last_became_peered":"2024-09-16T04:48:24.001074+0000","last_unstale":"2024-09-16T04:48:24.001315+0000","last_undegraded":"2024-09-16T04:48:23.981866+0000","last_fullsized":"2024-09-16T04:48:24.001315+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T04:48:24.003157+0000","last_change":"2024-09-16T04:48:24.003157+0000","last_active":"2024-09-16T04:48:24.003157+0000","last_peered":"2024-09-16T04:48:24.003157+0000","last_clean":"2024-09-16T04:48:24.003157+0000","last_became_active":"2024-09-16T04:48:24.002091+0000","last_became_peered":"2024-09-16T04:48:24.002091+0000","last_unstale":"2024-09-16T04:48:24.003157+0000","last_undegraded":"2024-09-16T04:48:24.003157+0000","last_fullsized":"2024-09-16T04:48:24.003157+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:19.944720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":39,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:23.998578+0000","last_change":"2024-09-16T04:48:23.999042+0000","last_active":"2024-09-16T04:48:23.998578+0000","last_peered":"2024-09-16T04:48:23.998578+0000","last_clean":"2024-09-16T04:48:11.121168+0000","last_became_active":"2024-09-16T04:48:23.998254+0000","last_became_peered":"2024-09-16T04:48:23.998254+0000","last_unstale":"2024-09-16T04:48:23.998578+0000","last_undegraded":"2024-09-16T04:48:23.980783+0000","last_fullsized":"2024-09-16T04:48:23.998578+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T04:48:23.991672+0000","last_change":"2024-09-16T04:48:23.991672+0000","last_active":"2024-09-16T04:48:23.991672+0000","last_peered":"2024-09-16T04:48:23.991672+0000","last_clean":"2024-09-16T04:48:23.991672+0000","last_became_active":"2024-09-16T04:48:23.990798+0000","last_became_peered":"2024-09-16T04:48:23.990798+0000","last_unstale":"2024-09-16T04:48:23.991672+0000","last_undegraded":"2024-09-16T04:48:23.991672+0000","last_fullsized":"2024-09-16T04:48:23.991672+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:16:31.289220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":35,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:23.998957+0000","last_change":"2024-09-16T04:48:23.999300+0000","last_active":"2024-09-16T04:48:23.998957+0000","last_peered":"2024-09-16T04:48:23.998957+0000","last_clean":"2024-09-16T04:48:00.428013+0000","last_became_active":"2024-09-16T04:48:23.998735+0000","last_became_peered":"2024-09-16T04:48:23.998735+0000","last_unstale":"2024-09-16T04:48:23.998957+0000","last_undegraded":"2024-09-16T04:48:23.981265+0000","last_fullsized":"2024-09-16T04:48:23.998957+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":50,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T04:48:24.001643+0000","last_change":"2024-09-16T04:48:24.001643+0000","last_active":"2024-09-16T04:48:24.001643+0000","last_peered":"2024-09-16T04:48:24.001643+0000","last_clean":"2024-09-16T04:48:24.001643+0000","last_became_active":"2024-09-16T04:48:24.000749+0000","last_became_peered":"2024-09-16T04:48:24.000749+0000","last_unstale":"2024-09-16T04:48:24.001643+0000","last_undegraded":"2024-09-16T04:48:24.001643+0000","last_fullsized":"2024-09-16T04:48:24.001643+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:31:23.143999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":54,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T04:48:24.002386+0000","last_change":"2024-09-16T04:48:24.002386+0000","last_active":"2024-09-16T04:48:24.002386+0000","last_peered":"2024-09-16T04:48:24.002386+0000","last_clean":"2024-09-16T04:48:24.002386+0000","last_became_active":"2024-09-16T04:48:24.001473+0000","last_became_peered":"2024-09-16T04:48:24.001473+0000","last_unstale":"2024-09-16T04:48:24.002386+0000","last_undegraded":"2024-09-16T04:48:24.002386+0000","last_fullsized":"2024-09-16T04:48:24.002386+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:57:37.950928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":43,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:24.001920+0000","last_change":"2024-09-16T04:48:24.002426+0000","last_active":"2024-09-16T04:48:24.001920+0000","last_peered":"2024-09-16T04:48:24.001920+0000","last_clean":"2024-09-16T04:48:10.711306+0000","last_became_active":"2024-09-16T04:48:24.001658+0000","last_became_peered":"2024-09-16T04:48:24.001658+0000","last_unstale":"2024-09-16T04:48:24.001920+0000","last_undegraded":"2024-09-16T04:48:23.981235+0000","last_fullsized":"2024-09-16T04:48:24.001920+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"0,1","objects":1},{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705678,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":81360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182403,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":260,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":266240,"data_stored":70254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:48:29.959 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:48:30.312 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:30.312 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":58,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":26,"num_osds":3,"num_up_osds":2,"osd_up_since":1726462102,"num_in_osds":2,"osd_in_since":1726462079,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+degraded","count":12},{"state_name":"active+clean","count":10},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56430592,"bytes_avail":193217097728,"bytes_total":193273528320,"degraded_objects":27,"degraded_total":60,"degraded_ratio":0.45000000000000001},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-16T04:48:00.250331+0000","services":{"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T04:48:30.328 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': 'b9a92eba-c518-4afa-ae65-ed2825dd0c06', 'health': {'status': 'HEALTH_WARN', 'checks': {'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded', 'count': 14}, 'muted': False}}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 58, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 26, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726462102, 'num_in_osds': 2, 'osd_in_since': 1726462079, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+recovery_wait+degraded', 'count': 12}, {'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovering+degraded', 'count': 2}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56430592, 'bytes_avail': 193217097728, 'bytes_total': 193273528320, 'degraded_objects': 27, 'degraded_total': 60, 'degraded_ratio': 0.45}, 'fsmap': {'epoch': 1, 'btime': '2024-09-16T04:47:27:159526+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 2, 'modified': '2024-09-16T04:48:00.250331+0000', 'services': {'mgr': {'daemons': {'summary': '', 'x': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}, 'mon': {'daemons': {'summary': '', 'a': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, 'b': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, 'c': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}, 'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '1': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-16T04:48:30.329 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-16T04:48:30.329 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T04:48:30.329 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T04:48:30.416 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T04:48:30.416 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-16T04:48:30.721 INFO:tasks.ceph.mon.c.smithi012.stderr:2024-09-16T04:48:30.717+0000 7fbe3b864640 -1 mon.c@2(leader).osd e26 definitely_dead 0 2024-09-16T04:48:31.190 INFO:teuthology.orchestra.run.smithi012.stderr:marked down osd.1. 2024-09-16T04:48:31.221 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-16T04:48:31.526 INFO:tasks.ceph.mon.c.smithi012.stderr:2024-09-16T04:48:31.521+0000 7fbe3b864640 -1 mon.c@2(leader).osd e27 definitely_dead 0 2024-09-16T04:48:32.207 INFO:teuthology.orchestra.run.smithi012.stderr:marked out osd.1. 2024-09-16T04:48:32.232 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd lost 1 --yes-i-really-mean-it 2024-09-16T04:48:33.212 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.205+0000 7f2e6592a640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-16T04:48:33.212 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.205+0000 7f2e64928640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-16T04:48:33.212 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.205+0000 7f2e63926640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-16T04:48:33.213 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.205+0000 7f2e65129640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-16T04:48:33.213 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.205+0000 7f2e6812f640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-16T04:48:33.213 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.205+0000 7f2e68930640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-16T04:48:33.214 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.205+0000 7f2e64127640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-16T04:48:33.214 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.205+0000 7f2e69932640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-16T04:48:33.214 INFO:teuthology.orchestra.run.smithi012.stderr:marked osd lost in epoch 27 2024-09-16T04:48:33.215 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.205+0000 7f2e65129640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-16T04:48:33.215 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.209+0000 7f2e6712d640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-16T04:48:33.215 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.209+0000 7f2e63125640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-16T04:48:33.215 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.209+0000 7f2e62924640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-16T04:48:33.215 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.209+0000 7f2e6692c640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-16T04:48:33.216 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:33.209+0000 7f2e62123640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-16T04:48:33.240 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T04:48:33.240 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T04:48:33.242 INFO:tasks.ceph.osd.2:Started 2024-09-16T04:48:33.243 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T04:48:33.367 INFO:teuthology.orchestra.run.smithi012.stderr:no valid command found; 10 closest matches: 2024-09-16T04:48:33.367 INFO:teuthology.orchestra.run.smithi012.stderr:0 2024-09-16T04:48:33.367 INFO:teuthology.orchestra.run.smithi012.stderr:1 2024-09-16T04:48:33.367 INFO:teuthology.orchestra.run.smithi012.stderr:2 2024-09-16T04:48:33.367 INFO:teuthology.orchestra.run.smithi012.stderr:abort 2024-09-16T04:48:33.367 INFO:teuthology.orchestra.run.smithi012.stderr:assert 2024-09-16T04:48:33.367 INFO:teuthology.orchestra.run.smithi012.stderr:config diff 2024-09-16T04:48:33.368 INFO:teuthology.orchestra.run.smithi012.stderr:config diff get 2024-09-16T04:48:33.368 INFO:teuthology.orchestra.run.smithi012.stderr:config get 2024-09-16T04:48:33.368 INFO:teuthology.orchestra.run.smithi012.stderr:config help [] 2024-09-16T04:48:33.368 INFO:teuthology.orchestra.run.smithi012.stderr:config set ... 2024-09-16T04:48:33.368 INFO:teuthology.orchestra.run.smithi012.stderr:admin_socket: invalid command 2024-09-16T04:48:33.369 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T04:48:33.369 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-16T04:48:34.795 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-16T04:48:34.789+0000 7faac72c57c0 -1 Falling back to public interface 2024-09-16T04:48:38.370 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T04:48:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:no valid command found; 10 closest matches: 2024-09-16T04:48:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:0 2024-09-16T04:48:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:1 2024-09-16T04:48:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2 2024-09-16T04:48:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:abort 2024-09-16T04:48:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:assert 2024-09-16T04:48:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:bluestore allocator dump block 2024-09-16T04:48:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:bluestore allocator fragmentation block 2024-09-16T04:48:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:bluestore allocator fragmentation histogram block [] [] 2024-09-16T04:48:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:bluestore allocator score block 2024-09-16T04:48:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:config diff 2024-09-16T04:48:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:admin_socket: invalid command 2024-09-16T04:48:38.504 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T04:48:38.505 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-16T04:48:39.435 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-16T04:48:39.429+0000 7faac72c57c0 -1 osd.2 18 log_to_monitors true 2024-09-16T04:48:40.392 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e65129640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-16T04:48:40.393 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e6592a640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-16T04:48:40.393 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e69932640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-16T04:48:40.393 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e64928640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-16T04:48:40.393 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e69131640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-16T04:48:40.393 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e68930640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-16T04:48:40.394 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e6812f640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-16T04:48:40.394 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e64127640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-16T04:48:40.394 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e63926640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-16T04:48:40.394 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e63125640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-16T04:48:40.394 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e6692c640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-16T04:48:40.394 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e62123640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-16T04:48:40.394 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e6712d640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-16T04:48:40.395 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:40.385+0000 7f2e62924640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-16T04:48:40.400 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-16T04:48:40.393+0000 7faabb092640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T04:48:41.390 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.381+0000 7f2e6592a640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-16T04:48:41.390 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.381+0000 7f2e69932640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-16T04:48:41.390 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.381+0000 7f2e65129640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-16T04:48:41.391 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.385+0000 7f2e6812f640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-16T04:48:41.391 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.385+0000 7f2e64928640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-16T04:48:41.391 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.385+0000 7f2e65129640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-16T04:48:41.392 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.385+0000 7f2e64127640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-16T04:48:41.392 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.385+0000 7f2e68930640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-16T04:48:41.392 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.385+0000 7f2e63926640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-16T04:48:41.392 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.385+0000 7f2e6712d640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-16T04:48:41.393 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.385+0000 7f2e63125640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-16T04:48:41.393 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.385+0000 7f2e62924640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-16T04:48:41.398 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.385+0000 7f2e6692c640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-16T04:48:41.399 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:41.385+0000 7f2e6612b640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-16T04:48:42.116 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:42.105+0000 7f2e63926640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-16T04:48:42.119 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:42.105+0000 7f2e68930640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-16T04:48:42.119 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:42.105+0000 7f2e69131640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-16T04:48:42.120 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:42.105+0000 7f2e6692c640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-16T04:48:42.121 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:42.105+0000 7f2e63125640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-16T04:48:42.121 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:42.105+0000 7f2e64928640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-16T04:48:42.122 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:42.109+0000 7f2e6612b640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-16T04:48:43.126 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:43.117+0000 7f2e69131640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-16T04:48:43.129 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:43.121+0000 7f2e6792e640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-16T04:48:43.129 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:43.121+0000 7f2e64928640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-16T04:48:43.130 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:43.121+0000 7f2e64928640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-16T04:48:43.130 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:43.121+0000 7f2e6692c640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-16T04:48:43.133 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:43.121+0000 7f2e62123640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-16T04:48:43.133 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:43.121+0000 7f2e63125640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-16T04:48:43.506 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T04:48:43.642 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:48:43.642 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-16T04:48:43.642 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-16T04:48:43.642 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:48:43.656 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-16T04:48:43.962 INFO:tasks.ceph.mon.c.smithi012.stderr:2024-09-16T04:48:43.957+0000 7fbe3b864640 -1 mon.c@2(leader).osd e33 definitely_dead 0 2024-09-16T04:48:44.418 INFO:teuthology.orchestra.run.smithi012.stderr:marked in osd.2. 2024-09-16T04:48:44.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.2 to be up 2024-09-16T04:48:44.446 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T04:48:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:44.744 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","created":"2024-09-16T04:47:27.160076+0000","modified":"2024-09-16T04:48:44.399308+0000","last_up_change":"2024-09-16T04:48:41.379829+0000","last_in_change":"2024-09-16T04:48:44.399308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-16T04:47:42.612259+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-16T04:47:55.339654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7fa4b8a1-d218-47cb-adc4-469674c829fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":21,"up_from":25,"up_thru":32,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6800","nonce":2560862748}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6801","nonce":2560862748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6803","nonce":2560862748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6802","nonce":2560862748}]},"public_addr":"172.21.15.12:6800/2560862748","cluster_addr":"172.21.15.12:6801/2560862748","heartbeat_back_addr":"172.21.15.12:6803/2560862748","heartbeat_front_addr":"172.21.15.12:6802/2560862748","state":["exists","up"]},{"osd":1,"uuid":"7234f3ae-2cf3-4dd4-8c21-4f170976dbf4","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6808","nonce":1033795119}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6809","nonce":1033795119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6811","nonce":1033795119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6810","nonce":1033795119}]},"public_addr":"172.21.15.12:6808/1033795119","cluster_addr":"172.21.15.12:6809/1033795119","heartbeat_back_addr":"172.21.15.12:6811/1033795119","heartbeat_front_addr":"172.21.15.12:6810/1033795119","state":["exists"]},{"osd":2,"uuid":"919a351e-ecee-4e06-bff1-ae68c80d320f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":18,"up_from":31,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6804","nonce":1526510468}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6805","nonce":1526510468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6807","nonce":1526510468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6806","nonce":1526510468}]},"public_addr":"172.21.15.12:6804/1526510468","cluster_addr":"172.21.15.12:6805/1526510468","heartbeat_back_addr":"172.21.15.12:6807/1526510468","heartbeat_front_addr":"172.21.15.12:6806/1526510468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.109010+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-16T04:47:35.949029+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.023975+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.1","osds":[0,2]},{"pgid":"1.3","osds":[0,2]},{"pgid":"1.5","osds":[0,2]},{"pgid":"1.7","osds":[0,2]},{"pgid":"2.0","osds":[0,2]},{"pgid":"2.1","osds":[0,2]},{"pgid":"2.3","osds":[0,2]},{"pgid":"2.c","osds":[0,2]},{"pgid":"2.d","osds":[0,2]},{"pgid":"2.e","osds":[0,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T04:48:44.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.2 is up 2024-09-16T04:48:44.763 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T04:48:44.764 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T04:48:44.922 INFO:teuthology.orchestra.run.smithi012.stdout:107374182407 2024-09-16T04:48:44.922 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:48:44.923 INFO:teuthology.orchestra.run.smithi012.stdout:133143986178 2024-09-16T04:48:44.923 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T04:48:45.255 INFO:teuthology.orchestra.run.smithi012.stdout:133143986177 2024-09-16T04:48:45.259 INFO:teuthology.orchestra.run.smithi012.stdout:107374182406 2024-09-16T04:48:45.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986178 got 133143986177 for osd.2 2024-09-16T04:48:45.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182407 got 107374182406 for osd.0 2024-09-16T04:48:46.275 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T04:48:46.278 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:48:46.451 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:46.445+0000 7f2e64928640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-16T04:48:46.451 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:46.445+0000 7f2e6792e640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-16T04:48:46.451 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:46.445+0000 7f2e68930640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-16T04:48:46.451 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:46.445+0000 7f2e69932640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-16T04:48:46.451 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:46.445+0000 7f2e62924640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-16T04:48:46.451 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:46.445+0000 7f2e62123640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-16T04:48:46.452 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:46.445+0000 7f2e63125640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-16T04:48:46.452 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:46.445+0000 7f2e69131640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-16T04:48:46.452 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-16T04:48:46.445+0000 7f2e6812f640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-16T04:48:46.593 INFO:teuthology.orchestra.run.smithi012.stdout:133143986178 2024-09-16T04:48:46.609 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986178 got 133143986178 for osd.2 2024-09-16T04:48:46.610 DEBUG:teuthology.parallel:result is None 2024-09-16T04:48:46.619 INFO:teuthology.orchestra.run.smithi012.stdout:107374182407 2024-09-16T04:48:46.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182407 got 107374182407 for osd.0 2024-09-16T04:48:46.635 DEBUG:teuthology.parallel:result is None 2024-09-16T04:48:46.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-16T04:48:46.635 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:48:46.966 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:46.966 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:48:46.984 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-09-16T04:48:46.443870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55616,"kb_used_data":604,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688064,"statfs":{"total":193273528320,"available":193216577536,"internally_reserved":0,"allocated":618496,"data_stored":183565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8877,"internal_metadata":56286547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.182459"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":37,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-16T04:48:43.128103+0000","last_change":"2024-09-16T04:48:44.427127+0000","last_active":"2024-09-16T04:48:44.427127+0000","last_peered":"2024-09-16T04:48:43.128103+0000","last_clean":"2024-09-16T04:48:23.997685+0000","last_became_active":"2024-09-16T04:48:32.209576+0000","last_became_peered":"2024-09-16T04:48:32.209576+0000","last_unstale":"2024-09-16T04:48:43.128103+0000","last_undegraded":"2024-09-16T04:48:43.128103+0000","last_fullsized":"2024-09-16T04:48:32.205733+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":40,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.434187+0000","last_change":"2024-09-16T04:48:45.415002+0000","last_active":"2024-09-16T04:48:42.113571+0000","last_peered":"2024-09-16T04:48:41.391666+0000","last_clean":"2024-09-16T04:48:10.022495+0000","last_became_active":"2024-09-16T04:48:32.210664+0000","last_became_peered":"2024-09-16T04:48:32.210664+0000","last_unstale":"2024-09-16T04:48:45.434187+0000","last_undegraded":"2024-09-16T04:48:45.434187+0000","last_fullsized":"2024-09-16T04:48:45.434187+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":54,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.433219+0000","last_change":"2024-09-16T04:48:45.418234+0000","last_active":"2024-09-16T04:48:42.112953+0000","last_peered":"2024-09-16T04:48:41.390865+0000","last_clean":"2024-09-16T04:48:10.874965+0000","last_became_active":"2024-09-16T04:48:32.210082+0000","last_became_peered":"2024-09-16T04:48:32.210082+0000","last_unstale":"2024-09-16T04:48:45.433219+0000","last_undegraded":"2024-09-16T04:48:45.433219+0000","last_fullsized":"2024-09-16T04:48:45.433219+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":39,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.416979+0000","last_change":"2024-09-16T04:48:45.416979+0000","last_active":"2024-09-16T04:48:42.115018+0000","last_peered":"2024-09-16T04:48:41.390505+0000","last_clean":"2024-09-16T04:48:23.996934+0000","last_became_active":"2024-09-16T04:48:32.209942+0000","last_became_peered":"2024-09-16T04:48:32.209942+0000","last_unstale":"2024-09-16T04:48:45.416979+0000","last_undegraded":"2024-09-16T04:48:45.416979+0000","last_fullsized":"2024-09-16T04:48:45.416979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":55,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-16T04:48:44.418548+0000","last_change":"2024-09-16T04:48:44.418548+0000","last_active":"2024-09-16T04:48:44.418548+0000","last_peered":"2024-09-16T04:48:43.128458+0000","last_clean":"2024-09-16T04:48:09.846075+0000","last_became_active":"2024-09-16T04:48:43.127839+0000","last_became_peered":"2024-09-16T04:48:43.127839+0000","last_unstale":"2024-09-16T04:48:44.418548+0000","last_undegraded":"2024-09-16T04:48:44.418548+0000","last_fullsized":"2024-09-16T04:48:44.418548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":50,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-16T04:48:43.125784+0000","last_change":"2024-09-16T04:48:44.421846+0000","last_active":"2024-09-16T04:48:44.421846+0000","last_peered":"2024-09-16T04:48:43.125784+0000","last_clean":"2024-09-16T04:48:11.202938+0000","last_became_active":"2024-09-16T04:48:32.212910+0000","last_became_peered":"2024-09-16T04:48:32.212910+0000","last_unstale":"2024-09-16T04:48:43.125784+0000","last_undegraded":"2024-09-16T04:48:32.206644+0000","last_fullsized":"2024-09-16T04:48:32.205733+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":51,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-16T04:48:43.122172+0000","last_change":"2024-09-16T04:48:44.422048+0000","last_active":"2024-09-16T04:48:44.422048+0000","last_peered":"2024-09-16T04:48:43.122172+0000","last_clean":"2024-09-16T04:48:10.270497+0000","last_became_active":"2024-09-16T04:48:32.212708+0000","last_became_peered":"2024-09-16T04:48:32.212708+0000","last_unstale":"2024-09-16T04:48:43.122172+0000","last_undegraded":"2024-09-16T04:48:32.206807+0000","last_fullsized":"2024-09-16T04:48:32.206614+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":53,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-16T04:48:44.420707+0000","last_change":"2024-09-16T04:48:44.420707+0000","last_active":"2024-09-16T04:48:44.420707+0000","last_peered":"2024-09-16T04:48:43.130319+0000","last_clean":"2024-09-16T04:47:58.315475+0000","last_became_active":"2024-09-16T04:48:43.129464+0000","last_became_peered":"2024-09-16T04:48:43.129464+0000","last_unstale":"2024-09-16T04:48:44.420707+0000","last_undegraded":"2024-09-16T04:48:44.420707+0000","last_fullsized":"2024-09-16T04:48:44.420707+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":61,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-16T04:48:43.121703+0000","last_change":"2024-09-16T04:48:44.425579+0000","last_active":"2024-09-16T04:48:44.425579+0000","last_peered":"2024-09-16T04:48:43.121703+0000","last_clean":"2024-09-16T04:48:00.427592+0000","last_became_active":"2024-09-16T04:48:32.208422+0000","last_became_peered":"2024-09-16T04:48:32.208422+0000","last_unstale":"2024-09-16T04:48:43.121703+0000","last_undegraded":"2024-09-16T04:48:43.121703+0000","last_fullsized":"2024-09-16T04:48:32.203108+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":56,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.430290+0000","last_change":"2024-09-16T04:48:45.416546+0000","last_active":"2024-09-16T04:48:42.115044+0000","last_peered":"2024-09-16T04:48:41.389212+0000","last_clean":"2024-09-16T04:48:10.956703+0000","last_became_active":"2024-09-16T04:48:32.212181+0000","last_became_peered":"2024-09-16T04:48:32.212181+0000","last_unstale":"2024-09-16T04:48:45.430290+0000","last_undegraded":"2024-09-16T04:48:45.430290+0000","last_fullsized":"2024-09-16T04:48:45.430290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.417591+0000","last_change":"2024-09-16T04:48:45.417591+0000","last_active":"2024-09-16T04:48:42.111628+0000","last_peered":"2024-09-16T04:48:41.387812+0000","last_clean":"2024-09-16T04:48:23.994030+0000","last_became_active":"2024-09-16T04:48:32.206830+0000","last_became_peered":"2024-09-16T04:48:32.206830+0000","last_unstale":"2024-09-16T04:48:45.417591+0000","last_undegraded":"2024-09-16T04:48:45.417591+0000","last_fullsized":"2024-09-16T04:48:45.417591+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":48,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-16T04:48:43.126511+0000","last_change":"2024-09-16T04:48:44.422373+0000","last_active":"2024-09-16T04:48:44.422373+0000","last_peered":"2024-09-16T04:48:43.126511+0000","last_clean":"2024-09-16T04:48:11.292504+0000","last_became_active":"2024-09-16T04:48:32.209589+0000","last_became_peered":"2024-09-16T04:48:32.209589+0000","last_unstale":"2024-09-16T04:48:43.126511+0000","last_undegraded":"2024-09-16T04:48:32.206192+0000","last_fullsized":"2024-09-16T04:48:32.206104+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":53,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-16T04:48:43.121344+0000","last_change":"2024-09-16T04:48:44.421254+0000","last_active":"2024-09-16T04:48:44.421254+0000","last_peered":"2024-09-16T04:48:43.121344+0000","last_clean":"2024-09-16T04:48:23.998001+0000","last_became_active":"2024-09-16T04:48:32.208648+0000","last_became_peered":"2024-09-16T04:48:32.208648+0000","last_unstale":"2024-09-16T04:48:43.121344+0000","last_undegraded":"2024-09-16T04:48:32.205405+0000","last_fullsized":"2024-09-16T04:48:32.205284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":44,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.428505+0000","last_change":"2024-09-16T04:48:45.418981+0000","last_active":"2024-09-16T04:48:42.110426+0000","last_peered":"2024-09-16T04:48:41.388589+0000","last_clean":"2024-09-16T04:48:09.768821+0000","last_became_active":"2024-09-16T04:48:32.208514+0000","last_became_peered":"2024-09-16T04:48:32.208514+0000","last_unstale":"2024-09-16T04:48:45.428505+0000","last_undegraded":"2024-09-16T04:48:45.428505+0000","last_fullsized":"2024-09-16T04:48:45.428505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":10,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.414659+0000","last_change":"2024-09-16T04:48:45.414659+0000","last_active":"2024-09-16T04:48:45.414659+0000","last_peered":"2024-09-16T04:47:44.338267+0000","last_clean":"2024-09-16T04:47:44.338267+0000","last_became_active":"2024-09-16T04:47:44.337472+0000","last_became_peered":"2024-09-16T04:47:44.337472+0000","last_unstale":"2024-09-16T04:48:45.414659+0000","last_undegraded":"2024-09-16T04:48:45.414659+0000","last_fullsized":"2024-09-16T04:48:45.414659+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":45,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.427664+0000","last_change":"2024-09-16T04:48:45.417247+0000","last_active":"2024-09-16T04:48:42.109980+0000","last_peered":"2024-09-16T04:48:41.388016+0000","last_clean":"2024-09-16T04:48:10.384545+0000","last_became_active":"2024-09-16T04:48:32.208254+0000","last_became_peered":"2024-09-16T04:48:32.208254+0000","last_unstale":"2024-09-16T04:48:45.427664+0000","last_undegraded":"2024-09-16T04:48:45.427664+0000","last_fullsized":"2024-09-16T04:48:45.427664+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":47,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-16T04:48:43.128146+0000","last_change":"2024-09-16T04:48:44.426074+0000","last_active":"2024-09-16T04:48:44.426074+0000","last_peered":"2024-09-16T04:48:43.128146+0000","last_clean":"2024-09-16T04:48:00.429368+0000","last_became_active":"2024-09-16T04:48:32.209132+0000","last_became_peered":"2024-09-16T04:48:32.209132+0000","last_unstale":"2024-09-16T04:48:43.128146+0000","last_undegraded":"2024-09-16T04:48:32.205453+0000","last_fullsized":"2024-09-16T04:48:32.205296+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":61,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-16T04:48:43.126184+0000","last_change":"2024-09-16T04:48:44.422863+0000","last_active":"2024-09-16T04:48:44.422863+0000","last_peered":"2024-09-16T04:48:43.126184+0000","last_clean":"2024-09-16T04:48:00.432877+0000","last_became_active":"2024-09-16T04:48:32.210195+0000","last_became_peered":"2024-09-16T04:48:32.210195+0000","last_unstale":"2024-09-16T04:48:43.126184+0000","last_undegraded":"2024-09-16T04:48:43.126184+0000","last_fullsized":"2024-09-16T04:48:32.205126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":52,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-16T04:48:43.126950+0000","last_change":"2024-09-16T04:48:44.425750+0000","last_active":"2024-09-16T04:48:44.425750+0000","last_peered":"2024-09-16T04:48:43.126950+0000","last_clean":"2024-09-16T04:48:11.121168+0000","last_became_active":"2024-09-16T04:48:32.210639+0000","last_became_peered":"2024-09-16T04:48:32.210639+0000","last_unstale":"2024-09-16T04:48:43.126950+0000","last_undegraded":"2024-09-16T04:48:32.206174+0000","last_fullsized":"2024-09-16T04:48:32.205869+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":40,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.415727+0000","last_change":"2024-09-16T04:48:45.415727+0000","last_active":"2024-09-16T04:48:42.110795+0000","last_peered":"2024-09-16T04:48:41.390681+0000","last_clean":"2024-09-16T04:48:23.994223+0000","last_became_active":"2024-09-16T04:48:32.208874+0000","last_became_peered":"2024-09-16T04:48:32.208874+0000","last_unstale":"2024-09-16T04:48:45.415727+0000","last_undegraded":"2024-09-16T04:48:45.415727+0000","last_fullsized":"2024-09-16T04:48:45.415727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":48,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-16T04:48:43.128266+0000","last_change":"2024-09-16T04:48:44.425581+0000","last_active":"2024-09-16T04:48:44.425581+0000","last_peered":"2024-09-16T04:48:43.128266+0000","last_clean":"2024-09-16T04:48:00.428013+0000","last_became_active":"2024-09-16T04:48:32.209548+0000","last_became_peered":"2024-09-16T04:48:32.209548+0000","last_unstale":"2024-09-16T04:48:43.128266+0000","last_undegraded":"2024-09-16T04:48:32.205675+0000","last_fullsized":"2024-09-16T04:48:32.205548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":61,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-16T04:48:43.126520+0000","last_change":"2024-09-16T04:48:44.425576+0000","last_active":"2024-09-16T04:48:44.425576+0000","last_peered":"2024-09-16T04:48:43.126520+0000","last_clean":"2024-09-16T04:48:00.428141+0000","last_became_active":"2024-09-16T04:48:32.208562+0000","last_became_peered":"2024-09-16T04:48:32.208562+0000","last_unstale":"2024-09-16T04:48:43.126520+0000","last_undegraded":"2024-09-16T04:48:43.126520+0000","last_fullsized":"2024-09-16T04:48:32.204564+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.414890+0000","last_change":"2024-09-16T04:48:45.414890+0000","last_active":"2024-09-16T04:48:42.113265+0000","last_peered":"2024-09-16T04:48:41.392232+0000","last_clean":"2024-09-16T04:48:00.440762+0000","last_became_active":"2024-09-16T04:48:32.208693+0000","last_became_peered":"2024-09-16T04:48:32.208693+0000","last_unstale":"2024-09-16T04:48:45.414890+0000","last_undegraded":"2024-09-16T04:48:45.414890+0000","last_fullsized":"2024-09-16T04:48:45.414890+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":57,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-16T04:48:43.125737+0000","last_change":"2024-09-16T04:48:44.422287+0000","last_active":"2024-09-16T04:48:44.422287+0000","last_peered":"2024-09-16T04:48:43.125737+0000","last_clean":"2024-09-16T04:48:10.711306+0000","last_became_active":"2024-09-16T04:48:32.208779+0000","last_became_peered":"2024-09-16T04:48:32.208779+0000","last_unstale":"2024-09-16T04:48:43.125737+0000","last_undegraded":"2024-09-16T04:48:32.204781+0000","last_fullsized":"2024-09-16T04:48:32.204465+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":288,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":294912,"data_stored":91819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":28176041},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182407,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":316,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":323584,"data_stored":91746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:48:46.986 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:48:47.347 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:47.347 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":75,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726462121,"num_in_osds":2,"osd_in_since":1726462124,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":14},{"state_name":"remapped+peering","count":10}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56950784,"bytes_avail":193216577536,"bytes_total":193273528320,"inactive_pgs_ratio":1},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-16T04:48:34.259242+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T04:48:47.364 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': 'b9a92eba-c518-4afa-ae65-ed2825dd0c06', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 75, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726462121, 'num_in_osds': 2, 'osd_in_since': 1726462124, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 14}, {'state_name': 'remapped+peering', 'count': 10}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56950784, 'bytes_avail': 193216577536, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 1}, 'fsmap': {'epoch': 1, 'btime': '2024-09-16T04:47:27:159526+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 4, 'modified': '2024-09-16T04:48:34.259242+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-16T04:48:50.365 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:48:50.660 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:50.660 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:48:50.676 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-16T04:48:50.264156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":18,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55640,"kb_used_data":620,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688040,"statfs":{"total":193273528320,"available":193216552960,"internally_reserved":0,"allocated":634880,"data_stored":189827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8877,"internal_metadata":56286547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-23,"num_objects_misplaced":0,"num_objects_unfound":-9,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001970"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449502+0000","last_change":"2024-09-16T04:48:45.444872+0000","last_active":"2024-09-16T04:48:46.449502+0000","last_peered":"2024-09-16T04:48:46.449502+0000","last_clean":"2024-09-16T04:48:46.449502+0000","last_became_active":"2024-09-16T04:48:45.444050+0000","last_became_peered":"2024-09-16T04:48:45.444050+0000","last_unstale":"2024-09-16T04:48:46.449502+0000","last_undegraded":"2024-09-16T04:48:46.449502+0000","last_fullsized":"2024-09-16T04:48:46.449502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:36.117906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":40,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.434187+0000","last_change":"2024-09-16T04:48:45.415002+0000","last_active":"2024-09-16T04:48:42.113571+0000","last_peered":"2024-09-16T04:48:41.391666+0000","last_clean":"2024-09-16T04:48:10.022495+0000","last_became_active":"2024-09-16T04:48:32.210664+0000","last_became_peered":"2024-09-16T04:48:32.210664+0000","last_unstale":"2024-09-16T04:48:45.434187+0000","last_undegraded":"2024-09-16T04:48:45.434187+0000","last_fullsized":"2024-09-16T04:48:45.434187+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":54,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.433219+0000","last_change":"2024-09-16T04:48:45.418234+0000","last_active":"2024-09-16T04:48:42.112953+0000","last_peered":"2024-09-16T04:48:41.390865+0000","last_clean":"2024-09-16T04:48:10.874965+0000","last_became_active":"2024-09-16T04:48:32.210082+0000","last_became_peered":"2024-09-16T04:48:32.210082+0000","last_unstale":"2024-09-16T04:48:45.433219+0000","last_undegraded":"2024-09-16T04:48:45.433219+0000","last_fullsized":"2024-09-16T04:48:45.433219+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":39,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.416979+0000","last_change":"2024-09-16T04:48:45.416979+0000","last_active":"2024-09-16T04:48:42.115018+0000","last_peered":"2024-09-16T04:48:41.390505+0000","last_clean":"2024-09-16T04:48:23.996934+0000","last_became_active":"2024-09-16T04:48:32.209942+0000","last_became_peered":"2024-09-16T04:48:32.209942+0000","last_unstale":"2024-09-16T04:48:45.416979+0000","last_undegraded":"2024-09-16T04:48:45.416979+0000","last_fullsized":"2024-09-16T04:48:45.416979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":64,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-16T04:48:46.451429+0000","last_change":"2024-09-16T04:48:46.451429+0000","last_active":"2024-09-16T04:48:46.451429+0000","last_peered":"2024-09-16T04:48:46.451429+0000","last_clean":"2024-09-16T04:48:09.846075+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:48:46.451429+0000","last_undegraded":"2024-09-16T04:48:46.451429+0000","last_fullsized":"2024-09-16T04:48:46.451429+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":58,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449185+0000","last_change":"2024-09-16T04:48:45.442482+0000","last_active":"2024-09-16T04:48:46.449185+0000","last_peered":"2024-09-16T04:48:46.449185+0000","last_clean":"2024-09-16T04:48:11.202938+0000","last_became_active":"2024-09-16T04:48:45.441846+0000","last_became_peered":"2024-09-16T04:48:45.441846+0000","last_unstale":"2024-09-16T04:48:46.449185+0000","last_undegraded":"2024-09-16T04:48:45.421972+0000","last_fullsized":"2024-09-16T04:48:46.449185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":59,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449867+0000","last_change":"2024-09-16T04:48:45.445456+0000","last_active":"2024-09-16T04:48:46.449867+0000","last_peered":"2024-09-16T04:48:46.449867+0000","last_clean":"2024-09-16T04:48:10.270497+0000","last_became_active":"2024-09-16T04:48:45.444639+0000","last_became_peered":"2024-09-16T04:48:45.444639+0000","last_unstale":"2024-09-16T04:48:46.449867+0000","last_undegraded":"2024-09-16T04:48:45.426169+0000","last_fullsized":"2024-09-16T04:48:46.449867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":61,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449403+0000","last_change":"2024-09-16T04:48:45.440763+0000","last_active":"2024-09-16T04:48:46.449403+0000","last_peered":"2024-09-16T04:48:46.449403+0000","last_clean":"2024-09-16T04:47:58.315475+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:48:46.449403+0000","last_undegraded":"2024-09-16T04:48:45.420366+0000","last_fullsized":"2024-09-16T04:48:46.449403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449263+0000","last_change":"2024-09-16T04:48:45.440924+0000","last_active":"2024-09-16T04:48:46.449263+0000","last_peered":"2024-09-16T04:48:46.449263+0000","last_clean":"2024-09-16T04:48:46.449263+0000","last_became_active":"2024-09-16T04:48:45.440123+0000","last_became_peered":"2024-09-16T04:48:45.440123+0000","last_unstale":"2024-09-16T04:48:46.449263+0000","last_undegraded":"2024-09-16T04:48:46.449263+0000","last_fullsized":"2024-09-16T04:48:46.449263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.338015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":56,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.430290+0000","last_change":"2024-09-16T04:48:45.416546+0000","last_active":"2024-09-16T04:48:42.115044+0000","last_peered":"2024-09-16T04:48:41.389212+0000","last_clean":"2024-09-16T04:48:10.956703+0000","last_became_active":"2024-09-16T04:48:32.212181+0000","last_became_peered":"2024-09-16T04:48:32.212181+0000","last_unstale":"2024-09-16T04:48:45.430290+0000","last_undegraded":"2024-09-16T04:48:45.430290+0000","last_fullsized":"2024-09-16T04:48:45.430290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.417591+0000","last_change":"2024-09-16T04:48:45.417591+0000","last_active":"2024-09-16T04:48:42.111628+0000","last_peered":"2024-09-16T04:48:41.387812+0000","last_clean":"2024-09-16T04:48:23.994030+0000","last_became_active":"2024-09-16T04:48:32.206830+0000","last_became_peered":"2024-09-16T04:48:32.206830+0000","last_unstale":"2024-09-16T04:48:45.417591+0000","last_undegraded":"2024-09-16T04:48:45.417591+0000","last_fullsized":"2024-09-16T04:48:45.417591+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":56,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449362+0000","last_change":"2024-09-16T04:48:45.445370+0000","last_active":"2024-09-16T04:48:46.449362+0000","last_peered":"2024-09-16T04:48:46.449362+0000","last_clean":"2024-09-16T04:48:11.292504+0000","last_became_active":"2024-09-16T04:48:45.444610+0000","last_became_peered":"2024-09-16T04:48:45.444610+0000","last_unstale":"2024-09-16T04:48:46.449362+0000","last_undegraded":"2024-09-16T04:48:45.422293+0000","last_fullsized":"2024-09-16T04:48:46.449362+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":61,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449124+0000","last_change":"2024-09-16T04:48:45.442094+0000","last_active":"2024-09-16T04:48:46.449124+0000","last_peered":"2024-09-16T04:48:46.449124+0000","last_clean":"2024-09-16T04:48:23.998001+0000","last_became_active":"2024-09-16T04:48:45.440291+0000","last_became_peered":"2024-09-16T04:48:45.440291+0000","last_unstale":"2024-09-16T04:48:46.449124+0000","last_undegraded":"2024-09-16T04:48:45.420971+0000","last_fullsized":"2024-09-16T04:48:46.449124+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":44,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.428505+0000","last_change":"2024-09-16T04:48:45.418981+0000","last_active":"2024-09-16T04:48:42.110426+0000","last_peered":"2024-09-16T04:48:41.388589+0000","last_clean":"2024-09-16T04:48:09.768821+0000","last_became_active":"2024-09-16T04:48:32.208514+0000","last_became_peered":"2024-09-16T04:48:32.208514+0000","last_unstale":"2024-09-16T04:48:45.428505+0000","last_undegraded":"2024-09-16T04:48:45.428505+0000","last_fullsized":"2024-09-16T04:48:45.428505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":10,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.414659+0000","last_change":"2024-09-16T04:48:45.414659+0000","last_active":"2024-09-16T04:48:45.414659+0000","last_peered":"2024-09-16T04:47:44.338267+0000","last_clean":"2024-09-16T04:47:44.338267+0000","last_became_active":"2024-09-16T04:47:44.337472+0000","last_became_peered":"2024-09-16T04:47:44.337472+0000","last_unstale":"2024-09-16T04:48:45.414659+0000","last_undegraded":"2024-09-16T04:48:45.414659+0000","last_fullsized":"2024-09-16T04:48:45.414659+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":45,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.427664+0000","last_change":"2024-09-16T04:48:45.417247+0000","last_active":"2024-09-16T04:48:42.109980+0000","last_peered":"2024-09-16T04:48:41.388016+0000","last_clean":"2024-09-16T04:48:10.384545+0000","last_became_active":"2024-09-16T04:48:32.208254+0000","last_became_peered":"2024-09-16T04:48:32.208254+0000","last_unstale":"2024-09-16T04:48:45.427664+0000","last_undegraded":"2024-09-16T04:48:45.427664+0000","last_fullsized":"2024-09-16T04:48:45.427664+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":55,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449463+0000","last_change":"2024-09-16T04:48:45.445513+0000","last_active":"2024-09-16T04:48:46.449463+0000","last_peered":"2024-09-16T04:48:46.449463+0000","last_clean":"2024-09-16T04:48:00.429368+0000","last_became_active":"2024-09-16T04:48:45.444748+0000","last_became_peered":"2024-09-16T04:48:45.444748+0000","last_unstale":"2024-09-16T04:48:46.449463+0000","last_undegraded":"2024-09-16T04:48:45.425112+0000","last_fullsized":"2024-09-16T04:48:46.449463+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449472+0000","last_change":"2024-09-16T04:48:45.443815+0000","last_active":"2024-09-16T04:48:46.449472+0000","last_peered":"2024-09-16T04:48:46.449472+0000","last_clean":"2024-09-16T04:48:46.449472+0000","last_became_active":"2024-09-16T04:48:45.443064+0000","last_became_peered":"2024-09-16T04:48:45.443064+0000","last_unstale":"2024-09-16T04:48:46.449472+0000","last_undegraded":"2024-09-16T04:48:46.449472+0000","last_fullsized":"2024-09-16T04:48:46.449472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:35.280081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":60,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449406+0000","last_change":"2024-09-16T04:48:45.446398+0000","last_active":"2024-09-16T04:48:46.449406+0000","last_peered":"2024-09-16T04:48:46.449406+0000","last_clean":"2024-09-16T04:48:11.121168+0000","last_became_active":"2024-09-16T04:48:45.445679+0000","last_became_peered":"2024-09-16T04:48:45.445679+0000","last_unstale":"2024-09-16T04:48:46.449406+0000","last_undegraded":"2024-09-16T04:48:45.426487+0000","last_fullsized":"2024-09-16T04:48:46.449406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":40,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.415727+0000","last_change":"2024-09-16T04:48:45.415727+0000","last_active":"2024-09-16T04:48:42.110795+0000","last_peered":"2024-09-16T04:48:41.390681+0000","last_clean":"2024-09-16T04:48:23.994223+0000","last_became_active":"2024-09-16T04:48:32.208874+0000","last_became_peered":"2024-09-16T04:48:32.208874+0000","last_unstale":"2024-09-16T04:48:45.415727+0000","last_undegraded":"2024-09-16T04:48:45.415727+0000","last_fullsized":"2024-09-16T04:48:45.415727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":56,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449638+0000","last_change":"2024-09-16T04:48:45.444871+0000","last_active":"2024-09-16T04:48:46.449638+0000","last_peered":"2024-09-16T04:48:46.449638+0000","last_clean":"2024-09-16T04:48:00.428013+0000","last_became_active":"2024-09-16T04:48:45.444067+0000","last_became_peered":"2024-09-16T04:48:45.444067+0000","last_unstale":"2024-09-16T04:48:46.449638+0000","last_undegraded":"2024-09-16T04:48:45.426010+0000","last_fullsized":"2024-09-16T04:48:46.449638+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449269+0000","last_change":"2024-09-16T04:48:45.437264+0000","last_active":"2024-09-16T04:48:46.449269+0000","last_peered":"2024-09-16T04:48:46.449269+0000","last_clean":"2024-09-16T04:48:46.449269+0000","last_became_active":"2024-09-16T04:48:45.435646+0000","last_became_peered":"2024-09-16T04:48:45.435646+0000","last_unstale":"2024-09-16T04:48:46.449269+0000","last_undegraded":"2024-09-16T04:48:46.449269+0000","last_fullsized":"2024-09-16T04:48:46.449269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:29.272297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":35,"state":"peering","last_fresh":"2024-09-16T04:48:45.414890+0000","last_change":"2024-09-16T04:48:45.414890+0000","last_active":"2024-09-16T04:48:42.113265+0000","last_peered":"2024-09-16T04:48:41.392232+0000","last_clean":"2024-09-16T04:48:00.440762+0000","last_became_active":"2024-09-16T04:48:32.208693+0000","last_became_peered":"2024-09-16T04:48:32.208693+0000","last_unstale":"2024-09-16T04:48:45.414890+0000","last_undegraded":"2024-09-16T04:48:45.414890+0000","last_fullsized":"2024-09-16T04:48:45.414890+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":65,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449190+0000","last_change":"2024-09-16T04:48:45.445172+0000","last_active":"2024-09-16T04:48:46.449190+0000","last_peered":"2024-09-16T04:48:46.449190+0000","last_clean":"2024-09-16T04:48:10.711306+0000","last_became_active":"2024-09-16T04:48:45.440990+0000","last_became_peered":"2024-09-16T04:48:45.440990+0000","last_unstale":"2024-09-16T04:48:46.449190+0000","last_undegraded":"2024-09-16T04:48:45.422071+0000","last_fullsized":"2024-09-16T04:48:46.449190+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":18,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":288,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":294912,"data_stored":91819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":28176041},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182408,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":332,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":339968,"data_stored":98008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:48:50.678 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:48:51.018 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:51.018 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"18/30 objects unfound (60.000%)","count":18},"muted":false},"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg peering","count":1},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34/60 objects degraded (56.667%), 9 pgs degraded","count":9},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":78,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726462121,"num_in_osds":2,"osd_in_since":1726462124,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":10},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+clean","count":4},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56975360,"bytes_avail":193216552960,"bytes_total":193273528320,"inactive_pgs_ratio":0.4166666567325592,"degraded_objects":34,"degraded_total":60,"degraded_ratio":0.56666666666666665,"unfound_objects":18,"unfound_total":30,"unfound_ratio":0.59999999999999998},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-16T04:48:34.259242+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T04:48:51.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': 'b9a92eba-c518-4afa-ae65-ed2825dd0c06', 'health': {'status': 'HEALTH_WARN', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '18/30 objects unfound (60.000%)', 'count': 18}, 'muted': False}, 'PG_AVAILABILITY': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Reduced data availability: 1 pg peering', 'count': 1}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 34/60 objects degraded (56.667%), 9 pgs degraded', 'count': 9}, 'muted': False}}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 78, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726462121, 'num_in_osds': 2, 'osd_in_since': 1726462124, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 10}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+clean', 'count': 4}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56975360, 'bytes_avail': 193216552960, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 0.4166666567325592, 'degraded_objects': 34, 'degraded_total': 60, 'degraded_ratio': 0.5666666666666667, 'unfound_objects': 18, 'unfound_total': 30, 'unfound_ratio': 0.6}, 'fsmap': {'epoch': 1, 'btime': '2024-09-16T04:47:27:159526+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 4, 'modified': '2024-09-16T04:48:34.259242+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-16T04:48:52.459 INFO:tasks.ceph.mon.c.smithi012.stderr:2024-09-16T04:48:52.453+0000 7fbe3e069640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 5 pgs recovery_unfound (PG_DAMAGED) 2024-09-16T04:48:54.037 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:48:54.345 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:54.384 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:48:54.385 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-09-16T04:48:54.265622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55652,"kb_used_data":636,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688028,"statfs":{"total":193273528320,"available":193216540672,"internally_reserved":0,"allocated":651264,"data_stored":195245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8877,"internal_metadata":56286547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.854870"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449502+0000","last_change":"2024-09-16T04:48:45.444872+0000","last_active":"2024-09-16T04:48:46.449502+0000","last_peered":"2024-09-16T04:48:46.449502+0000","last_clean":"2024-09-16T04:48:46.449502+0000","last_became_active":"2024-09-16T04:48:45.444050+0000","last_became_peered":"2024-09-16T04:48:45.444050+0000","last_unstale":"2024-09-16T04:48:46.449502+0000","last_undegraded":"2024-09-16T04:48:46.449502+0000","last_fullsized":"2024-09-16T04:48:46.449502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:36.117906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":48,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-16T04:48:46.457915+0000","last_change":"2024-09-16T04:48:46.458102+0000","last_active":"2024-09-16T04:48:46.457915+0000","last_peered":"2024-09-16T04:48:46.457915+0000","last_clean":"2024-09-16T04:48:10.022495+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:48:46.457915+0000","last_undegraded":"2024-09-16T04:48:46.446954+0000","last_fullsized":"2024-09-16T04:48:46.457915+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":62,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-16T04:48:46.458465+0000","last_change":"2024-09-16T04:48:46.458693+0000","last_active":"2024-09-16T04:48:46.458465+0000","last_peered":"2024-09-16T04:48:46.458465+0000","last_clean":"2024-09-16T04:48:10.874965+0000","last_became_active":"2024-09-16T04:48:46.453489+0000","last_became_peered":"2024-09-16T04:48:46.453489+0000","last_unstale":"2024-09-16T04:48:46.458465+0000","last_undegraded":"2024-09-16T04:48:46.445832+0000","last_fullsized":"2024-09-16T04:48:46.458465+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.454392+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:48:46.454392+0000","last_peered":"2024-09-16T04:48:46.454392+0000","last_clean":"2024-09-16T04:48:46.454392+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:48:46.454392+0000","last_undegraded":"2024-09-16T04:48:46.454392+0000","last_fullsized":"2024-09-16T04:48:46.454392+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":64,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-16T04:48:46.451429+0000","last_change":"2024-09-16T04:48:46.451429+0000","last_active":"2024-09-16T04:48:46.451429+0000","last_peered":"2024-09-16T04:48:46.451429+0000","last_clean":"2024-09-16T04:48:09.846075+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:48:46.451429+0000","last_undegraded":"2024-09-16T04:48:46.451429+0000","last_fullsized":"2024-09-16T04:48:46.451429+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":58,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449185+0000","last_change":"2024-09-16T04:48:45.442482+0000","last_active":"2024-09-16T04:48:46.449185+0000","last_peered":"2024-09-16T04:48:46.449185+0000","last_clean":"2024-09-16T04:48:11.202938+0000","last_became_active":"2024-09-16T04:48:45.441846+0000","last_became_peered":"2024-09-16T04:48:45.441846+0000","last_unstale":"2024-09-16T04:48:46.449185+0000","last_undegraded":"2024-09-16T04:48:45.421972+0000","last_fullsized":"2024-09-16T04:48:46.449185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":59,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449867+0000","last_change":"2024-09-16T04:48:45.445456+0000","last_active":"2024-09-16T04:48:46.449867+0000","last_peered":"2024-09-16T04:48:46.449867+0000","last_clean":"2024-09-16T04:48:10.270497+0000","last_became_active":"2024-09-16T04:48:45.444639+0000","last_became_peered":"2024-09-16T04:48:45.444639+0000","last_unstale":"2024-09-16T04:48:46.449867+0000","last_undegraded":"2024-09-16T04:48:45.426169+0000","last_fullsized":"2024-09-16T04:48:46.449867+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":61,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449403+0000","last_change":"2024-09-16T04:48:45.440763+0000","last_active":"2024-09-16T04:48:46.449403+0000","last_peered":"2024-09-16T04:48:46.449403+0000","last_clean":"2024-09-16T04:47:58.315475+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:48:46.449403+0000","last_undegraded":"2024-09-16T04:48:45.420366+0000","last_fullsized":"2024-09-16T04:48:46.449403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449263+0000","last_change":"2024-09-16T04:48:45.440924+0000","last_active":"2024-09-16T04:48:46.449263+0000","last_peered":"2024-09-16T04:48:46.449263+0000","last_clean":"2024-09-16T04:48:46.449263+0000","last_became_active":"2024-09-16T04:48:45.440123+0000","last_became_peered":"2024-09-16T04:48:45.440123+0000","last_unstale":"2024-09-16T04:48:46.449263+0000","last_undegraded":"2024-09-16T04:48:46.449263+0000","last_fullsized":"2024-09-16T04:48:46.449263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.338015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":64,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-16T04:48:46.459650+0000","last_change":"2024-09-16T04:48:46.459927+0000","last_active":"2024-09-16T04:48:46.459650+0000","last_peered":"2024-09-16T04:48:46.459650+0000","last_clean":"2024-09-16T04:48:10.956703+0000","last_became_active":"2024-09-16T04:48:46.454004+0000","last_became_peered":"2024-09-16T04:48:46.454004+0000","last_unstale":"2024-09-16T04:48:46.459650+0000","last_undegraded":"2024-09-16T04:48:46.446159+0000","last_fullsized":"2024-09-16T04:48:46.459650+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.454453+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:48:46.454453+0000","last_peered":"2024-09-16T04:48:46.454453+0000","last_clean":"2024-09-16T04:48:46.454453+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:48:46.454453+0000","last_undegraded":"2024-09-16T04:48:46.454453+0000","last_fullsized":"2024-09-16T04:48:46.454453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":56,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449362+0000","last_change":"2024-09-16T04:48:45.445370+0000","last_active":"2024-09-16T04:48:46.449362+0000","last_peered":"2024-09-16T04:48:46.449362+0000","last_clean":"2024-09-16T04:48:11.292504+0000","last_became_active":"2024-09-16T04:48:45.444610+0000","last_became_peered":"2024-09-16T04:48:45.444610+0000","last_unstale":"2024-09-16T04:48:46.449362+0000","last_undegraded":"2024-09-16T04:48:45.422293+0000","last_fullsized":"2024-09-16T04:48:46.449362+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":61,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449124+0000","last_change":"2024-09-16T04:48:45.442094+0000","last_active":"2024-09-16T04:48:46.449124+0000","last_peered":"2024-09-16T04:48:46.449124+0000","last_clean":"2024-09-16T04:48:23.998001+0000","last_became_active":"2024-09-16T04:48:45.440291+0000","last_became_peered":"2024-09-16T04:48:45.440291+0000","last_unstale":"2024-09-16T04:48:46.449124+0000","last_undegraded":"2024-09-16T04:48:45.420971+0000","last_fullsized":"2024-09-16T04:48:46.449124+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":52,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-16T04:48:46.455538+0000","last_change":"2024-09-16T04:48:46.455713+0000","last_active":"2024-09-16T04:48:46.455538+0000","last_peered":"2024-09-16T04:48:46.455538+0000","last_clean":"2024-09-16T04:48:09.768821+0000","last_became_active":"2024-09-16T04:48:46.452651+0000","last_became_peered":"2024-09-16T04:48:46.452651+0000","last_unstale":"2024-09-16T04:48:46.455538+0000","last_undegraded":"2024-09-16T04:48:46.445318+0000","last_fullsized":"2024-09-16T04:48:46.455538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.454753+0000","last_change":"2024-09-16T04:48:46.454753+0000","last_active":"2024-09-16T04:48:46.454753+0000","last_peered":"2024-09-16T04:48:46.454753+0000","last_clean":"2024-09-16T04:48:46.454753+0000","last_became_active":"2024-09-16T04:48:46.454400+0000","last_became_peered":"2024-09-16T04:48:46.454400+0000","last_unstale":"2024-09-16T04:48:46.454753+0000","last_undegraded":"2024-09-16T04:48:46.454753+0000","last_fullsized":"2024-09-16T04:48:46.454753+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:50:39.528170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":53,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-16T04:48:46.459049+0000","last_change":"2024-09-16T04:48:46.459274+0000","last_active":"2024-09-16T04:48:46.459049+0000","last_peered":"2024-09-16T04:48:46.459049+0000","last_clean":"2024-09-16T04:48:10.384545+0000","last_became_active":"2024-09-16T04:48:46.454141+0000","last_became_peered":"2024-09-16T04:48:46.454141+0000","last_unstale":"2024-09-16T04:48:46.459049+0000","last_undegraded":"2024-09-16T04:48:46.444952+0000","last_fullsized":"2024-09-16T04:48:46.459049+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":55,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449463+0000","last_change":"2024-09-16T04:48:45.445513+0000","last_active":"2024-09-16T04:48:46.449463+0000","last_peered":"2024-09-16T04:48:46.449463+0000","last_clean":"2024-09-16T04:48:00.429368+0000","last_became_active":"2024-09-16T04:48:45.444748+0000","last_became_peered":"2024-09-16T04:48:45.444748+0000","last_unstale":"2024-09-16T04:48:46.449463+0000","last_undegraded":"2024-09-16T04:48:45.425112+0000","last_fullsized":"2024-09-16T04:48:46.449463+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449472+0000","last_change":"2024-09-16T04:48:45.443815+0000","last_active":"2024-09-16T04:48:46.449472+0000","last_peered":"2024-09-16T04:48:46.449472+0000","last_clean":"2024-09-16T04:48:46.449472+0000","last_became_active":"2024-09-16T04:48:45.443064+0000","last_became_peered":"2024-09-16T04:48:45.443064+0000","last_unstale":"2024-09-16T04:48:46.449472+0000","last_undegraded":"2024-09-16T04:48:46.449472+0000","last_fullsized":"2024-09-16T04:48:46.449472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:35.280081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":60,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449406+0000","last_change":"2024-09-16T04:48:45.446398+0000","last_active":"2024-09-16T04:48:46.449406+0000","last_peered":"2024-09-16T04:48:46.449406+0000","last_clean":"2024-09-16T04:48:11.121168+0000","last_became_active":"2024-09-16T04:48:45.445679+0000","last_became_peered":"2024-09-16T04:48:45.445679+0000","last_unstale":"2024-09-16T04:48:46.449406+0000","last_undegraded":"2024-09-16T04:48:45.426487+0000","last_fullsized":"2024-09-16T04:48:46.449406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.453526+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:48:46.453526+0000","last_peered":"2024-09-16T04:48:46.453526+0000","last_clean":"2024-09-16T04:48:46.453526+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:48:46.453526+0000","last_undegraded":"2024-09-16T04:48:46.453526+0000","last_fullsized":"2024-09-16T04:48:46.453526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":56,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449638+0000","last_change":"2024-09-16T04:48:45.444871+0000","last_active":"2024-09-16T04:48:46.449638+0000","last_peered":"2024-09-16T04:48:46.449638+0000","last_clean":"2024-09-16T04:48:00.428013+0000","last_became_active":"2024-09-16T04:48:45.444067+0000","last_became_peered":"2024-09-16T04:48:45.444067+0000","last_unstale":"2024-09-16T04:48:46.449638+0000","last_undegraded":"2024-09-16T04:48:45.426010+0000","last_fullsized":"2024-09-16T04:48:46.449638+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449269+0000","last_change":"2024-09-16T04:48:45.437264+0000","last_active":"2024-09-16T04:48:46.449269+0000","last_peered":"2024-09-16T04:48:46.449269+0000","last_clean":"2024-09-16T04:48:46.449269+0000","last_became_active":"2024-09-16T04:48:45.435646+0000","last_became_peered":"2024-09-16T04:48:45.435646+0000","last_unstale":"2024-09-16T04:48:46.449269+0000","last_undegraded":"2024-09-16T04:48:46.449269+0000","last_fullsized":"2024-09-16T04:48:46.449269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:29.272297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.453655+0000","last_change":"2024-09-16T04:48:46.453655+0000","last_active":"2024-09-16T04:48:46.453655+0000","last_peered":"2024-09-16T04:48:46.453655+0000","last_clean":"2024-09-16T04:48:46.453655+0000","last_became_active":"2024-09-16T04:48:46.453374+0000","last_became_peered":"2024-09-16T04:48:46.453374+0000","last_unstale":"2024-09-16T04:48:46.453655+0000","last_undegraded":"2024-09-16T04:48:46.453655+0000","last_fullsized":"2024-09-16T04:48:46.453655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:49:42.225940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":65,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449190+0000","last_change":"2024-09-16T04:48:45.445172+0000","last_active":"2024-09-16T04:48:46.449190+0000","last_peered":"2024-09-16T04:48:46.449190+0000","last_clean":"2024-09-16T04:48:10.711306+0000","last_became_active":"2024-09-16T04:48:45.440990+0000","last_became_peered":"2024-09-16T04:48:45.440990+0000","last_unstale":"2024-09-16T04:48:46.449190+0000","last_undegraded":"2024-09-16T04:48:45.422071+0000","last_fullsized":"2024-09-16T04:48:46.449190+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986180,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":296,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":303104,"data_stored":94528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":28176041},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182409,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":340,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":348160,"data_stored":100717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:48:54.386 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:48:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:54.747 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 52/60 objects degraded (86.667%), 9 pgs degraded","count":9},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":82,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726462121,"num_in_osds":2,"osd_in_since":1726462124,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":5},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56987648,"bytes_avail":193216540672,"bytes_total":193273528320,"degraded_objects":52,"degraded_total":60,"degraded_ratio":0.8666666666666667,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-16T04:48:34.259242+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T04:48:54.765 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': 'b9a92eba-c518-4afa-ae65-ed2825dd0c06', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 52/60 objects degraded (86.667%), 9 pgs degraded', 'count': 9}, 'muted': False}}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 82, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726462121, 'num_in_osds': 2, 'osd_in_since': 1726462124, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 5}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56987648, 'bytes_avail': 193216540672, 'bytes_total': 193273528320, 'degraded_objects': 52, 'degraded_total': 60, 'degraded_ratio': 0.8666666666666667, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-16T04:47:27:159526+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 4, 'modified': '2024-09-16T04:48:34.259242+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-16T04:48:54.766 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-16T04:48:54.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T04:48:54.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T04:48:54.926 INFO:teuthology.orchestra.run.smithi012.stdout:133143986181 2024-09-16T04:48:54.927 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T04:48:54.936 INFO:teuthology.orchestra.run.smithi012.stdout:107374182410 2024-09-16T04:48:54.937 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:48:55.216 INFO:teuthology.orchestra.run.smithi012.stdout:133143986180 2024-09-16T04:48:55.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986181 got 133143986180 for osd.2 2024-09-16T04:48:55.265 INFO:teuthology.orchestra.run.smithi012.stdout:107374182409 2024-09-16T04:48:55.282 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182410 got 107374182409 for osd.0 2024-09-16T04:48:56.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T04:48:56.283 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:48:56.543 INFO:teuthology.orchestra.run.smithi012.stdout:133143986181 2024-09-16T04:48:56.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986181 got 133143986181 for osd.2 2024-09-16T04:48:56.560 DEBUG:teuthology.parallel:result is None 2024-09-16T04:48:56.577 INFO:teuthology.orchestra.run.smithi012.stdout:107374182410 2024-09-16T04:48:56.593 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182410 got 107374182410 for osd.0 2024-09-16T04:48:56.594 DEBUG:teuthology.parallel:result is None 2024-09-16T04:48:56.594 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:48:56.984 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:48:56.984 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 52/60 objects degraded (86.667%), 9 pgs degraded","count":9},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":84,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726462121,"num_in_osds":2,"osd_in_since":1726462124,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":5},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56987648,"bytes_avail":193216540672,"bytes_total":193273528320,"degraded_objects":52,"degraded_total":60,"degraded_ratio":0.8666666666666667,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-16T04:48:34.259242+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T04:48:57.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': 'b9a92eba-c518-4afa-ae65-ed2825dd0c06', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 52/60 objects degraded (86.667%), 9 pgs degraded', 'count': 9}, 'muted': False}}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 84, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726462121, 'num_in_osds': 2, 'osd_in_since': 1726462124, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 5}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56987648, 'bytes_avail': 193216540672, 'bytes_total': 193273528320, 'degraded_objects': 52, 'degraded_total': 60, 'degraded_ratio': 0.8666666666666667, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-16T04:47:27:159526+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 4, 'modified': '2024-09-16T04:48:34.259242+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-16T04:48:57.001 INFO:tasks.rep_lost_unfound_delete:there are 27 unfound objects 2024-09-16T04:48:57.001 DEBUG:teuthology.orchestra.run.smithi012:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.admin -b 4096 -p unfounddel_pool -t 20 bench 240 write' 2024-09-16T04:48:57.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:hints = 1 2024-09-16T04:48:57.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Maintaining 20 concurrent writes of 4096 bytes to objects of size 4096 for up to 240 seconds or 0 objects 2024-09-16T04:48:57.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Object prefix: benchmark_data_smithi012_34635 2024-09-16T04:48:57.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T04:48:57.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 0 0 0 0 0 0 - 0 2024-09-16T04:48:58.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 1 20 2732 2712 10.5932 10.5938 0.00631444 0.00734254 2024-09-16T04:48:59.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 2 19 5378 5359 10.4656 10.3398 0.0103909 0.00744808 2024-09-16T04:49:00.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 3 20 7936 7916 10.3063 9.98828 0.00351715 0.00756585 2024-09-16T04:49:01.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 4 20 10333 10313 10.0695 9.36328 0.00582382 0.00775035 2024-09-16T04:49:02.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 5 20 12498 12478 9.74691 8.45703 0.0100448 0.00800655 2024-09-16T04:49:03.046 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 6 20 14632 14612 9.51163 8.33594 0.0077326 0.00820508 2024-09-16T04:49:04.047 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 7 19 16823 16804 9.3751 8.5625 0.00691431 0.00832444 2024-09-16T04:49:05.047 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 8 20 18900 18880 9.21683 8.10938 0.00925322 0.00846848 2024-09-16T04:49:06.047 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 9 20 20958 20938 9.08591 8.03906 0.00837706 0.00859063 2024-09-16T04:49:07.004 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:07.047 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 10 20 22873 22853 8.92533 7.48047 0.00769927 0.00874556 2024-09-16T04:49:07.537 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:07.538 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:07.566 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-16T04:49:06.269044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48718735,"num_objects":11920,"num_object_clones":0,"num_object_copies":23840,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":11920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11947,"num_write_kb":47608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11948,"ondisk_log_size":11948,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":161580,"kb_used_data":85772,"kb_used_omap":8,"kb_used_meta":75767,"kb_avail":188582100,"statfs":{"total":193273528320,"available":193108070400,"internally_reserved":0,"allocated":87830528,"data_stored":87374509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8877,"internal_metadata":77585747},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,13,4,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48701440,"num_objects":11890,"num_object_clones":0,"num_object_copies":23780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11890,"num_write_kb":47560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003421"},"pg_stats":[{"pgid":"2.f","version":"36'783","reported_seq":830,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:01.968333+0000","last_change":"2024-09-16T04:48:45.444872+0000","last_active":"2024-09-16T04:49:01.968333+0000","last_peered":"2024-09-16T04:49:01.968333+0000","last_clean":"2024-09-16T04:49:01.968333+0000","last_became_active":"2024-09-16T04:48:45.444050+0000","last_became_peered":"2024-09-16T04:48:45.444050+0000","last_unstale":"2024-09-16T04:49:01.968333+0000","last_undegraded":"2024-09-16T04:49:01.968333+0000","last_fullsized":"2024-09-16T04:49:01.968333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:36.117906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3207168,"num_objects":783,"num_object_clones":0,"num_object_copies":1566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":783,"num_write_kb":3132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'704","reported_seq":746,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-16T04:49:01.430801+0000","last_change":"2024-09-16T04:48:57.056523+0000","last_active":"2024-09-16T04:49:01.430801+0000","last_peered":"2024-09-16T04:49:01.430801+0000","last_clean":"2024-09-16T04:48:10.022495+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:01.430801+0000","last_undegraded":"2024-09-16T04:48:46.458102+0000","last_fullsized":"2024-09-16T04:49:01.430801+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2868434,"num_objects":702,"num_object_clones":0,"num_object_copies":1404,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":2804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":702}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'653","reported_seq":711,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-16T04:49:01.425784+0000","last_change":"2024-09-16T04:48:57.056145+0000","last_active":"2024-09-16T04:49:01.425784+0000","last_peered":"2024-09-16T04:49:01.425784+0000","last_clean":"2024-09-16T04:48:10.874965+0000","last_became_active":"2024-09-16T04:48:46.453489+0000","last_became_peered":"2024-09-16T04:48:46.453489+0000","last_unstale":"2024-09-16T04:49:01.425784+0000","last_undegraded":"2024-09-16T04:48:46.458693+0000","last_fullsized":"2024-09-16T04:49:01.425784+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2663017,"num_objects":651,"num_object_clones":0,"num_object_copies":1302,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":653,"num_write_kb":2602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":651}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'679","reported_seq":725,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:01.417268+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:01.417268+0000","last_peered":"2024-09-16T04:49:01.417268+0000","last_clean":"2024-09-16T04:49:01.417268+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:01.417268+0000","last_undegraded":"2024-09-16T04:49:01.417268+0000","last_fullsized":"2024-09-16T04:49:01.417268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2781184,"num_objects":679,"num_object_clones":0,"num_object_copies":1358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":679,"num_write_kb":2716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'752","reported_seq":811,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-16T04:49:01.969174+0000","last_change":"2024-09-16T04:48:46.451429+0000","last_active":"2024-09-16T04:49:01.969174+0000","last_peered":"2024-09-16T04:49:01.969174+0000","last_clean":"2024-09-16T04:48:09.846075+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:01.969174+0000","last_undegraded":"2024-09-16T04:49:01.969174+0000","last_fullsized":"2024-09-16T04:49:01.969174+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":752,"log_dups_size":0,"ondisk_log_size":752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3060946,"num_objects":749,"num_object_clones":0,"num_object_copies":1498,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":752,"num_write_kb":2992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":749}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'772","reported_seq":825,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:01.966762+0000","last_change":"2024-09-16T04:48:45.442482+0000","last_active":"2024-09-16T04:49:01.966762+0000","last_peered":"2024-09-16T04:49:01.966762+0000","last_clean":"2024-09-16T04:48:11.202938+0000","last_became_active":"2024-09-16T04:48:45.441846+0000","last_became_peered":"2024-09-16T04:48:45.441846+0000","last_unstale":"2024-09-16T04:49:01.966762+0000","last_undegraded":"2024-09-16T04:48:45.421972+0000","last_fullsized":"2024-09-16T04:49:01.966762+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3151058,"num_objects":771,"num_object_clones":0,"num_object_copies":1542,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":772,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":771}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'803","reported_seq":857,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:01.970752+0000","last_change":"2024-09-16T04:48:45.445456+0000","last_active":"2024-09-16T04:49:01.970752+0000","last_peered":"2024-09-16T04:49:01.970752+0000","last_clean":"2024-09-16T04:48:10.270497+0000","last_became_active":"2024-09-16T04:48:45.444639+0000","last_became_peered":"2024-09-16T04:48:45.444639+0000","last_unstale":"2024-09-16T04:49:01.970752+0000","last_undegraded":"2024-09-16T04:48:45.426169+0000","last_fullsized":"2024-09-16T04:49:01.970752+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3273938,"num_objects":801,"num_object_clones":0,"num_object_copies":1602,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":803,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":801}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'756","reported_seq":816,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:01.922278+0000","last_change":"2024-09-16T04:48:45.440763+0000","last_active":"2024-09-16T04:49:01.922278+0000","last_peered":"2024-09-16T04:49:01.922278+0000","last_clean":"2024-09-16T04:47:58.315475+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:01.922278+0000","last_undegraded":"2024-09-16T04:48:45.420366+0000","last_fullsized":"2024-09-16T04:49:01.922278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3093097,"num_objects":756,"num_object_clones":0,"num_object_copies":1512,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":756,"num_write_kb":3021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":756}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449263+0000","last_change":"2024-09-16T04:48:45.440924+0000","last_active":"2024-09-16T04:48:46.449263+0000","last_peered":"2024-09-16T04:48:46.449263+0000","last_clean":"2024-09-16T04:48:46.449263+0000","last_became_active":"2024-09-16T04:48:45.440123+0000","last_became_peered":"2024-09-16T04:48:45.440123+0000","last_unstale":"2024-09-16T04:48:46.449263+0000","last_undegraded":"2024-09-16T04:48:46.449263+0000","last_fullsized":"2024-09-16T04:48:46.449263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.338015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'723","reported_seq":782,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-16T04:49:01.416525+0000","last_change":"2024-09-16T04:48:57.056433+0000","last_active":"2024-09-16T04:49:01.416525+0000","last_peered":"2024-09-16T04:49:01.416525+0000","last_clean":"2024-09-16T04:48:10.956703+0000","last_became_active":"2024-09-16T04:48:46.454004+0000","last_became_peered":"2024-09-16T04:48:46.454004+0000","last_unstale":"2024-09-16T04:49:01.416525+0000","last_undegraded":"2024-09-16T04:48:46.459927+0000","last_fullsized":"2024-09-16T04:49:01.416525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2942162,"num_objects":720,"num_object_clones":0,"num_object_copies":1440,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":723,"num_write_kb":2876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":720}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.454453+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:48:46.454453+0000","last_peered":"2024-09-16T04:48:46.454453+0000","last_clean":"2024-09-16T04:48:46.454453+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:48:46.454453+0000","last_undegraded":"2024-09-16T04:48:46.454453+0000","last_fullsized":"2024-09-16T04:48:46.454453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'814","reported_seq":864,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:01.968151+0000","last_change":"2024-09-16T04:48:45.445370+0000","last_active":"2024-09-16T04:49:01.968151+0000","last_peered":"2024-09-16T04:49:01.968151+0000","last_clean":"2024-09-16T04:48:11.292504+0000","last_became_active":"2024-09-16T04:48:45.444610+0000","last_became_peered":"2024-09-16T04:48:45.444610+0000","last_unstale":"2024-09-16T04:49:01.968151+0000","last_undegraded":"2024-09-16T04:48:45.422293+0000","last_fullsized":"2024-09-16T04:49:01.968151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3310802,"num_objects":810,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":814,"num_write_kb":3236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":810}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":61,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449124+0000","last_change":"2024-09-16T04:48:45.442094+0000","last_active":"2024-09-16T04:48:46.449124+0000","last_peered":"2024-09-16T04:48:46.449124+0000","last_clean":"2024-09-16T04:48:23.998001+0000","last_became_active":"2024-09-16T04:48:45.440291+0000","last_became_peered":"2024-09-16T04:48:45.440291+0000","last_unstale":"2024-09-16T04:48:46.449124+0000","last_undegraded":"2024-09-16T04:48:45.420971+0000","last_fullsized":"2024-09-16T04:48:46.449124+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'676","reported_seq":725,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-16T04:49:01.418951+0000","last_change":"2024-09-16T04:48:57.056444+0000","last_active":"2024-09-16T04:49:01.418951+0000","last_peered":"2024-09-16T04:49:01.418951+0000","last_clean":"2024-09-16T04:48:09.768821+0000","last_became_active":"2024-09-16T04:48:46.452651+0000","last_became_peered":"2024-09-16T04:48:46.452651+0000","last_unstale":"2024-09-16T04:49:01.418951+0000","last_undegraded":"2024-09-16T04:48:46.455713+0000","last_fullsized":"2024-09-16T04:49:01.418951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2757842,"num_objects":675,"num_object_clones":0,"num_object_copies":1350,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":676,"num_write_kb":2695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":675}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.454753+0000","last_change":"2024-09-16T04:48:46.454753+0000","last_active":"2024-09-16T04:48:46.454753+0000","last_peered":"2024-09-16T04:48:46.454753+0000","last_clean":"2024-09-16T04:48:46.454753+0000","last_became_active":"2024-09-16T04:48:46.454400+0000","last_became_peered":"2024-09-16T04:48:46.454400+0000","last_unstale":"2024-09-16T04:48:46.454753+0000","last_undegraded":"2024-09-16T04:48:46.454753+0000","last_fullsized":"2024-09-16T04:48:46.454753+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:50:39.528170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'686","reported_seq":735,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-16T04:49:01.431744+0000","last_change":"2024-09-16T04:48:57.073198+0000","last_active":"2024-09-16T04:49:01.431744+0000","last_peered":"2024-09-16T04:49:01.431744+0000","last_clean":"2024-09-16T04:48:10.384545+0000","last_became_active":"2024-09-16T04:48:46.454141+0000","last_became_peered":"2024-09-16T04:48:46.454141+0000","last_unstale":"2024-09-16T04:49:01.431744+0000","last_undegraded":"2024-09-16T04:48:46.459274+0000","last_fullsized":"2024-09-16T04:49:01.431744+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2794706,"num_objects":684,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":686,"num_write_kb":2731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":684}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'758","reported_seq":812,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:01.954067+0000","last_change":"2024-09-16T04:48:45.445513+0000","last_active":"2024-09-16T04:49:01.954067+0000","last_peered":"2024-09-16T04:49:01.954067+0000","last_clean":"2024-09-16T04:48:00.429368+0000","last_became_active":"2024-09-16T04:48:45.444748+0000","last_became_peered":"2024-09-16T04:48:45.444748+0000","last_unstale":"2024-09-16T04:49:01.954067+0000","last_undegraded":"2024-09-16T04:48:45.425112+0000","last_fullsized":"2024-09-16T04:49:01.954067+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3101289,"num_objects":758,"num_object_clones":0,"num_object_copies":1516,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":758,"num_write_kb":3029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":758}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449472+0000","last_change":"2024-09-16T04:48:45.443815+0000","last_active":"2024-09-16T04:48:46.449472+0000","last_peered":"2024-09-16T04:48:46.449472+0000","last_clean":"2024-09-16T04:48:46.449472+0000","last_became_active":"2024-09-16T04:48:45.443064+0000","last_became_peered":"2024-09-16T04:48:45.443064+0000","last_unstale":"2024-09-16T04:48:46.449472+0000","last_undegraded":"2024-09-16T04:48:46.449472+0000","last_fullsized":"2024-09-16T04:48:46.449472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:35.280081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'754","reported_seq":809,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:01.965099+0000","last_change":"2024-09-16T04:48:45.446398+0000","last_active":"2024-09-16T04:49:01.965099+0000","last_peered":"2024-09-16T04:49:01.965099+0000","last_clean":"2024-09-16T04:48:11.121168+0000","last_became_active":"2024-09-16T04:48:45.445679+0000","last_became_peered":"2024-09-16T04:48:45.445679+0000","last_unstale":"2024-09-16T04:49:01.965099+0000","last_undegraded":"2024-09-16T04:48:45.426487+0000","last_fullsized":"2024-09-16T04:49:01.965099+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3069755,"num_objects":752,"num_object_clones":0,"num_object_copies":1504,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":754,"num_write_kb":3000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":752}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.453526+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:48:46.453526+0000","last_peered":"2024-09-16T04:48:46.453526+0000","last_clean":"2024-09-16T04:48:46.453526+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:48:46.453526+0000","last_undegraded":"2024-09-16T04:48:46.453526+0000","last_fullsized":"2024-09-16T04:48:46.453526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'728","reported_seq":782,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:01.952302+0000","last_change":"2024-09-16T04:48:45.444871+0000","last_active":"2024-09-16T04:49:01.952302+0000","last_peered":"2024-09-16T04:49:01.952302+0000","last_clean":"2024-09-16T04:48:00.428013+0000","last_became_active":"2024-09-16T04:48:45.444067+0000","last_became_peered":"2024-09-16T04:48:45.444067+0000","last_unstale":"2024-09-16T04:49:01.952302+0000","last_undegraded":"2024-09-16T04:48:45.426010+0000","last_fullsized":"2024-09-16T04:49:01.952302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2974930,"num_objects":728,"num_object_clones":0,"num_object_copies":1456,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":728,"num_write_kb":2906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":728}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449269+0000","last_change":"2024-09-16T04:48:45.437264+0000","last_active":"2024-09-16T04:48:46.449269+0000","last_peered":"2024-09-16T04:48:46.449269+0000","last_clean":"2024-09-16T04:48:46.449269+0000","last_became_active":"2024-09-16T04:48:45.435646+0000","last_became_peered":"2024-09-16T04:48:45.435646+0000","last_unstale":"2024-09-16T04:48:46.449269+0000","last_undegraded":"2024-09-16T04:48:46.449269+0000","last_fullsized":"2024-09-16T04:48:46.449269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:29.272297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.453655+0000","last_change":"2024-09-16T04:48:46.453655+0000","last_active":"2024-09-16T04:48:46.453655+0000","last_peered":"2024-09-16T04:48:46.453655+0000","last_clean":"2024-09-16T04:48:46.453655+0000","last_became_active":"2024-09-16T04:48:46.453374+0000","last_became_peered":"2024-09-16T04:48:46.453374+0000","last_unstale":"2024-09-16T04:48:46.453655+0000","last_undegraded":"2024-09-16T04:48:46.453655+0000","last_fullsized":"2024-09-16T04:48:46.453655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:49:42.225940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'904","reported_seq":959,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:01.974504+0000","last_change":"2024-09-16T04:48:45.445172+0000","last_active":"2024-09-16T04:49:01.974504+0000","last_peered":"2024-09-16T04:49:01.974504+0000","last_clean":"2024-09-16T04:48:10.711306+0000","last_became_active":"2024-09-16T04:48:45.440990+0000","last_became_peered":"2024-09-16T04:48:45.440990+0000","last_unstale":"2024-09-16T04:49:01.974504+0000","last_undegraded":"2024-09-16T04:48:45.422071+0000","last_fullsized":"2024-09-16T04:49:01.974504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3668388,"num_objects":899,"num_object_clones":0,"num_object_copies":1798,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":904,"num_write_kb":3587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":898}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":48718716,"num_objects":11918,"num_object_clones":0,"num_object_copies":23836,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":11918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11945,"num_write_kb":47606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96440320,"data_stored":96308118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11945,"ondisk_log_size":11945,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986183,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84004,"kb_used_data":43988,"kb_used_omap":4,"kb_used_meta":39995,"kb_avail":94287836,"statfs":{"total":96636764160,"available":96550744064,"internally_reserved":0,"allocated":45043712,"data_stored":44835136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":40955561},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,5,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182412,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77576,"kb_used_data":41784,"kb_used_omap":4,"kb_used_meta":35771,"kb_avail":94294264,"statfs":{"total":96636764160,"available":96557326336,"internally_reserved":0,"allocated":42786816,"data_stored":42539373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":36630186},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,8,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":50671616,"data_stored":50636826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45654016,"data_stored":45654016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:07.567 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.f 2024-09-16T04:49:07.568 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.e state active+recovery_unfound+degraded 2024-09-16T04:49:07.568 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.e list_unfound '{}' 2024-09-16T04:49:07.812 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:07.812 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:07.812 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:07.812 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:07.812 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:07.812 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:07.812 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:07.812 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:07.812 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:07.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:07.813 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 2, 2024-09-16T04:49:07.813 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 2, 2024-09-16T04:49:07.813 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:07.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:07.813 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:07.813 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existing_4", 2024-09-16T04:49:07.813 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:07.813 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:07.813 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 1224592654, 2024-09-16T04:49:07.813 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:07.813 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:07.813 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:07.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:07.814 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'3", 2024-09-16T04:49:07.814 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "21'1", 2024-09-16T04:49:07.814 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:07.814 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-16T04:49:07.814 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:07.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:07.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:07.814 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:07.814 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existing_6", 2024-09-16T04:49:07.814 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:07.814 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:07.814 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 3101443614, 2024-09-16T04:49:07.815 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:07.815 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:07.815 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:07.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:07.815 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'4", 2024-09-16T04:49:07.815 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "21'2", 2024-09-16T04:49:07.815 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:07.815 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-16T04:49:07.815 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:07.815 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:07.815 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:07.815 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "NotRecovering", 2024-09-16T04:49:07.816 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:07.816 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:07.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:07.816 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:07.816 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:07.816 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:07.816 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:07.816 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:07.816 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:07.835 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.e on osd.2 2024-09-16T04:49:07.835 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.e mark_unfound_lost delete 2024-09-16T04:49:08.047 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 11 20 24454 24434 8.67534 6.17578 0.0122981 0.00899721 2024-09-16T04:49:08.077 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-16T04:49:08.102 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.d state active+recovery_unfound+degraded 2024-09-16T04:49:08.102 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.d list_unfound '{}' 2024-09-16T04:49:08.347 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:08.347 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 1, 2024-09-16T04:49:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 1, 2024-09-16T04:49:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:08.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existed_7", 2024-09-16T04:49:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 2358579165, 2024-09-16T04:49:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'3", 2024-09-16T04:49:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:08.349 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "NotRecovering", 2024-09-16T04:49:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:08.350 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:08.350 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:08.369 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.d on osd.2 2024-09-16T04:49:08.369 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.d mark_unfound_lost delete 2024-09-16T04:49:08.610 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-16T04:49:08.647 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.c 2024-09-16T04:49:08.647 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.b state active+recovering 2024-09-16T04:49:08.648 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.b list_unfound '{}' 2024-09-16T04:49:08.894 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:08.894 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 2, 2024-09-16T04:49:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 2, 2024-09-16T04:49:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:08.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existed_3", 2024-09-16T04:49:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 943960171, 2024-09-16T04:49:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'5", 2024-09-16T04:49:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:08.896 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existing_1", 2024-09-16T04:49:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 2885884443, 2024-09-16T04:49:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:08.897 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'4", 2024-09-16T04:49:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "21'1", 2024-09-16T04:49:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-16T04:49:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "Active", 2024-09-16T04:49:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:08.898 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:08.899 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:08.899 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:08.932 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.b on osd.0 2024-09-16T04:49:08.932 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.b mark_unfound_lost delete 2024-09-16T04:49:09.047 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 12 20 26004 25984 8.45693 6.05469 0.0188211 0.00922749 2024-09-16T04:49:09.184 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-16T04:49:09.213 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.a state active+recovery_wait+degraded 2024-09-16T04:49:09.214 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.a list_unfound '{}' 2024-09-16T04:49:09.447 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:09.447 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 2, 2024-09-16T04:49:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 2, 2024-09-16T04:49:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existing_9", 2024-09-16T04:49:09.448 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 1217469130, 2024-09-16T04:49:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'3", 2024-09-16T04:49:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "21'1", 2024-09-16T04:49:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-16T04:49:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:09.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "new_9", 2024-09-16T04:49:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 170713514, 2024-09-16T04:49:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'2", 2024-09-16T04:49:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:09.450 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "Active", 2024-09-16T04:49:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:09.451 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:09.451 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:09.491 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.a on osd.0 2024-09-16T04:49:09.491 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.a mark_unfound_lost delete 2024-09-16T04:49:09.799 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-16T04:49:09.823 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.9 state active+recovery_wait+degraded 2024-09-16T04:49:09.824 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.9 list_unfound '{}' 2024-09-16T04:49:10.047 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 13 20 27556 27536 8.27272 6.0625 0.00992763 0.0094366 2024-09-16T04:49:10.077 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:10.077 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:10.078 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:10.078 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:10.078 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:10.078 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:10.078 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:10.078 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:10.078 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:10.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:10.078 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 2, 2024-09-16T04:49:10.078 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 2, 2024-09-16T04:49:10.078 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:10.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:10.078 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:10.079 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existing_5", 2024-09-16T04:49:10.079 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:10.079 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:10.079 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 2998549577, 2024-09-16T04:49:10.079 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:10.079 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:10.079 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:10.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:10.079 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'4", 2024-09-16T04:49:10.079 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "21'2", 2024-09-16T04:49:10.079 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:10.079 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-16T04:49:10.079 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:10.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:10.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:10.080 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:10.080 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existing_2", 2024-09-16T04:49:10.080 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:10.080 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:10.080 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 4032093161, 2024-09-16T04:49:10.080 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:10.080 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:10.080 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:10.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:10.080 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'3", 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "21'1", 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "Active", 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:10.081 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:10.082 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:10.082 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:10.106 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.9 on osd.0 2024-09-16T04:49:10.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.9 mark_unfound_lost delete 2024-09-16T04:49:10.405 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-16T04:49:10.445 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.8 state active+recovery_wait+degraded 2024-09-16T04:49:10.446 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.8 list_unfound '{}' 2024-09-16T04:49:10.656 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:10.656 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:10.656 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:10.656 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:10.656 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:10.656 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:10.656 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:10.656 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:10.657 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:10.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:10.657 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 1, 2024-09-16T04:49:10.657 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 1, 2024-09-16T04:49:10.657 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:10.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:10.657 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:10.657 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "new_8", 2024-09-16T04:49:10.657 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:10.657 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:10.657 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 2730327960, 2024-09-16T04:49:10.657 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:10.657 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'1", 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "Active", 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:10.658 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:10.659 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:10.659 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:10.659 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:10.659 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:10.659 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:10.683 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.8 on osd.0 2024-09-16T04:49:10.683 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.8 mark_unfound_lost delete 2024-09-16T04:49:10.914 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-16T04:49:10.949 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.0 2024-09-16T04:49:10.949 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.3 state active+recovery_unfound+degraded 2024-09-16T04:49:10.949 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.3 list_unfound '{}' 2024-09-16T04:49:11.047 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 14 19 29108 29089 8.1151 6.06641 0.0185686 0.00961912 2024-09-16T04:49:11.154 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:11.154 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:11.154 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:11.154 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:11.154 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:11.154 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:11.154 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:11.154 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:11.154 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:11.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 2, 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 2, 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existing_8", 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 1901787523, 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:11.155 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'5", 2024-09-16T04:49:11.156 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "21'3", 2024-09-16T04:49:11.156 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:11.156 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-16T04:49:11.156 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:11.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:11.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:11.156 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:11.156 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existed_1", 2024-09-16T04:49:11.156 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:11.156 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:11.156 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 3122379363, 2024-09-16T04:49:11.156 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:11.156 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'4", 2024-09-16T04:49:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "NotRecovering", 2024-09-16T04:49:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:11.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:11.158 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:11.158 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:11.158 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:11.158 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:11.158 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:11.158 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:11.179 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.3 on osd.2 2024-09-16T04:49:11.180 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.3 mark_unfound_lost delete 2024-09-16T04:49:11.439 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-16T04:49:11.463 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.1 2024-09-16T04:49:11.464 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.2 state active+recovery_wait+degraded 2024-09-16T04:49:11.464 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.2 list_unfound '{}' 2024-09-16T04:49:11.670 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:11.671 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:11.671 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:11.671 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:11.671 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:11.671 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:11.671 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:11.671 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:11.671 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:11.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:11.671 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 2, 2024-09-16T04:49:11.671 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 2, 2024-09-16T04:49:11.671 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:11.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existed_6", 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 1708673474, 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'5", 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:11.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:11.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:11.673 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:11.673 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existed_9", 2024-09-16T04:49:11.673 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:11.673 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:11.673 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 831402034, 2024-09-16T04:49:11.673 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:11.673 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:11.673 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:11.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:11.673 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'6", 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "Active", 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:11.674 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:11.675 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:11.675 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:11.695 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.2 on osd.0 2024-09-16T04:49:11.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.2 mark_unfound_lost delete 2024-09-16T04:49:11.953 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-16T04:49:11.990 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.2 2024-09-16T04:49:11.990 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.1 state active+recovery_unfound+degraded 2024-09-16T04:49:11.991 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.1 list_unfound '{}' 2024-09-16T04:49:12.047 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 15 20 30592 30572 7.96027 5.79297 0.0122869 0.00980567 2024-09-16T04:49:12.257 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:12.258 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:12.258 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:12.258 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:12.258 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:12.258 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:12.258 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:12.258 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:12.258 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:12.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:12.258 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 2, 2024-09-16T04:49:12.259 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 2, 2024-09-16T04:49:12.259 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:12.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:12.259 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:12.259 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existing_3", 2024-09-16T04:49:12.259 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:12.259 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:12.259 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 459768081, 2024-09-16T04:49:12.259 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:12.259 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:12.259 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:12.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:12.259 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'3", 2024-09-16T04:49:12.260 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "21'1", 2024-09-16T04:49:12.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:12.260 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-16T04:49:12.260 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:12.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:12.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:12.260 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:12.260 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "new_3", 2024-09-16T04:49:12.260 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:12.260 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:12.260 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 480221105, 2024-09-16T04:49:12.260 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:12.260 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:12.261 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:12.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:12.261 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'2", 2024-09-16T04:49:12.261 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:12.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:12.261 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:12.261 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:12.261 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:12.261 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:12.261 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "NotRecovering", 2024-09-16T04:49:12.261 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:12.261 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:12.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:12.262 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:12.262 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:12.262 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:12.262 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:12.262 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:12.262 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:12.294 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.1 on osd.2 2024-09-16T04:49:12.294 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.1 mark_unfound_lost delete 2024-09-16T04:49:12.460 INFO:tasks.ceph.mon.c.smithi012.stderr:2024-09-16T04:49:12.452+0000 7fbe3e069640 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 2 pgs recovery_unfound (PG_DAMAGED) 2024-09-16T04:49:12.551 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-16T04:49:12.583 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.3 2024-09-16T04:49:12.583 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.0 state active+recovery_unfound+degraded 2024-09-16T04:49:12.583 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.0 list_unfound '{}' 2024-09-16T04:49:12.803 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 2, 2024-09-16T04:49:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 2, 2024-09-16T04:49:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "new_5", 2024-09-16T04:49:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 4248945696, 2024-09-16T04:49:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'3", 2024-09-16T04:49:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existed_5", 2024-09-16T04:49:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 4212789968, 2024-09-16T04:49:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'4", 2024-09-16T04:49:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "NotRecovering", 2024-09-16T04:49:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:12.807 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:12.827 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.0 on osd.2 2024-09-16T04:49:12.827 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.0 mark_unfound_lost delete 2024-09-16T04:49:13.048 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 16 20 32042 32022 7.81674 5.66406 0.0108019 0.00998775 2024-09-16T04:49:13.109 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-16T04:49:13.139 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.7 state active+recovery_wait+degraded 2024-09-16T04:49:13.139 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.7 list_unfound '{}' 2024-09-16T04:49:13.374 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:13.374 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:13.374 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:13.374 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:13.374 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:13.374 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 1, 2024-09-16T04:49:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 1, 2024-09-16T04:49:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "new_7", 2024-09-16T04:49:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:13.375 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 3761017527, 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'1", 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "Active", 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:13.376 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:13.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:13.377 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:13.377 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:13.377 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:13.377 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:13.377 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:13.377 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:13.399 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.7 on osd.0 2024-09-16T04:49:13.399 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.7 mark_unfound_lost delete 2024-09-16T04:49:13.647 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-16T04:49:13.677 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.4 2024-09-16T04:49:13.677 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.6 state active+recovery_wait+degraded 2024-09-16T04:49:13.677 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.6 list_unfound '{}' 2024-09-16T04:49:13.913 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:13.913 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:13.913 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:13.913 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:13.913 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:13.913 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:13.913 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:13.913 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 3, 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 3, 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existed_8", 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 1861424966, 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:13.914 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:13.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:13.915 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'5", 2024-09-16T04:49:13.915 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:13.915 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:13.915 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:13.915 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:13.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:13.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:13.915 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:13.915 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "new_6", 2024-09-16T04:49:13.915 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:13.915 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:13.916 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 3910131350, 2024-09-16T04:49:13.916 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:13.916 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:13.916 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:13.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:13.916 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'4", 2024-09-16T04:49:13.916 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:13.916 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:13.916 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:13.916 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:13.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:13.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:13.916 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:13.917 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "new_4", 2024-09-16T04:49:13.917 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:13.917 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:13.917 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 2787839990, 2024-09-16T04:49:13.917 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:13.917 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:13.917 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:13.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:13.917 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'3", 2024-09-16T04:49:13.917 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:13.917 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:13.917 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:13.917 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:13.918 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:13.918 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:13.918 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "Active", 2024-09-16T04:49:13.918 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:13.918 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:13.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:13.918 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:13.918 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:13.918 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:13.918 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:13.918 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:13.918 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:13.941 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.6 on osd.0 2024-09-16T04:49:13.941 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.6 mark_unfound_lost delete 2024-09-16T04:49:14.049 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 17 20 33438 33418 7.67718 5.45312 0.0118579 0.010169 2024-09-16T04:49:14.219 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-16T04:49:14.268 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.5 2024-09-16T04:49:14.268 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.5 state active+recovery_wait+degraded 2024-09-16T04:49:14.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.5 list_unfound '{}' 2024-09-16T04:49:14.535 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 2, 2024-09-16T04:49:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 2, 2024-09-16T04:49:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "new_1", 2024-09-16T04:49:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 26281365, 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'1", 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "new_2", 2024-09-16T04:49:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 3196340981, 2024-09-16T04:49:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'2", 2024-09-16T04:49:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "Active", 2024-09-16T04:49:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:14.539 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:14.569 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.5 on osd.0 2024-09-16T04:49:14.569 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.5 mark_unfound_lost delete 2024-09-16T04:49:14.798 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-16T04:49:14.822 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.6 2024-09-16T04:49:14.822 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.7 2024-09-16T04:49:14.822 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.4 state active+recovery_wait+degraded 2024-09-16T04:49:14.822 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.4 list_unfound '{}' 2024-09-16T04:49:15.049 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 18 20 34804 34784 7.5471 5.33594 0.0130866 0.0103452 2024-09-16T04:49:15.064 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "offset": { 2024-09-16T04:49:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "", 2024-09-16T04:49:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": 0, 2024-09-16T04:49:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 0, 2024-09-16T04:49:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": -9223372036854775808, 2024-09-16T04:49:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "num_missing": 4, 2024-09-16T04:49:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "num_unfound": 3, 2024-09-16T04:49:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "objects": [ 2024-09-16T04:49:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existed_4", 2024-09-16T04:49:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 873891716, 2024-09-16T04:49:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'8", 2024-09-16T04:49:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existing_7", 2024-09-16T04:49:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 3996155460, 2024-09-16T04:49:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'9", 2024-09-16T04:49:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "21'6", 2024-09-16T04:49:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-16T04:49:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": { 2024-09-16T04:49:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "oid": "existed_2", 2024-09-16T04:49:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-16T04:49:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "snapid": -2, 2024-09-16T04:49:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "hash": 1767667508, 2024-09-16T04:49:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 0, 2024-09-16T04:49:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-16T04:49:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "namespace": "" 2024-09-16T04:49:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "need": "23'7", 2024-09-16T04:49:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "have": "0'0", 2024-09-16T04:49:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "none", 2024-09-16T04:49:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-16T04:49:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "locations": [] 2024-09-16T04:49:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "Active", 2024-09-16T04:49:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "available_might_have_unfound": true, 2024-09-16T04:49:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: "might_have_unfound": [ 2024-09-16T04:49:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": "1", 2024-09-16T04:49:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "osd is down" 2024-09-16T04:49:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: "more": false 2024-09-16T04:49:15.075 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:15.090 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.4 on osd.0 2024-09-16T04:49:15.091 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.4 mark_unfound_lost delete 2024-09-16T04:49:15.340 INFO:teuthology.orchestra.run.smithi012.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-16T04:49:15.371 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 5 2024-09-16T04:49:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-16T04:49:15.738 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 5 2024-09-16T04:49:15.960 INFO:teuthology.orchestra.run.smithi012.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-16T04:49:15.992 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T04:49:15.992 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T04:49:16.049 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 19 20 36203 36183 7.4375 5.46484 0.00772648 0.0104962 2024-09-16T04:49:16.364 INFO:teuthology.orchestra.run.smithi012.stdout:133143986186 2024-09-16T04:49:16.364 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T04:49:16.424 INFO:teuthology.orchestra.run.smithi012.stdout:107374182415 2024-09-16T04:49:16.424 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:49:16.953 INFO:teuthology.orchestra.run.smithi012.stdout:133143986185 2024-09-16T04:49:16.997 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986186 got 133143986185 for osd.2 2024-09-16T04:49:17.049 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:2024-09-16T04:49:17.048744+0000 min lat: 0.00206892 max lat: 0.0601159 avg lat: 0.010666 2024-09-16T04:49:17.049 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T04:49:17.049 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 20 20 37512 37492 7.32129 5.11328 0.0118635 0.010666 2024-09-16T04:49:17.145 INFO:teuthology.orchestra.run.smithi012.stdout:107374182414 2024-09-16T04:49:17.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182415 got 107374182414 for osd.0 2024-09-16T04:49:17.999 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T04:49:18.049 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 21 19 39012 38993 7.25184 5.86328 0.010374 0.0107657 2024-09-16T04:49:18.174 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T04:49:18.814 INFO:teuthology.orchestra.run.smithi012.stdout:133143986187 2024-09-16T04:49:18.851 INFO:teuthology.orchestra.run.smithi012.stdout:107374182416 2024-09-16T04:49:18.855 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986186 got 133143986187 for osd.2 2024-09-16T04:49:18.855 DEBUG:teuthology.parallel:result is None 2024-09-16T04:49:18.893 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182415 got 107374182416 for osd.0 2024-09-16T04:49:18.893 DEBUG:teuthology.parallel:result is None 2024-09-16T04:49:18.893 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-16T04:49:18.894 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:19.049 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 22 20 40243 40223 7.1406 4.80469 0.00804216 0.0109344 2024-09-16T04:49:19.490 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:19.490 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:19.529 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-16T04:49:18.271757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":152175503,"num_objects":37178,"num_object_clones":0,"num_object_copies":74356,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37205,"num_write_kb":148640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37233,"ondisk_log_size":37233,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":507956,"kb_used_data":291924,"kb_used_omap":8,"kb_used_meta":215991,"kb_avail":188235724,"statfs":{"total":193273528320,"available":192753381376,"internally_reserved":0,"allocated":298930176,"data_stored":298488073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8877,"internal_metadata":221175123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,7,2,8,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":103456768,"num_objects":25258,"num_object_clones":0,"num_object_copies":50516,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":25258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25258,"num_write_kb":101032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002713"},"pg_stats":[{"pgid":"2.f","version":"36'2375","reported_seq":2422,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.953856+0000","last_change":"2024-09-16T04:48:45.444872+0000","last_active":"2024-09-16T04:49:16.953856+0000","last_peered":"2024-09-16T04:49:16.953856+0000","last_clean":"2024-09-16T04:49:16.953856+0000","last_became_active":"2024-09-16T04:48:45.444050+0000","last_became_peered":"2024-09-16T04:48:45.444050+0000","last_unstale":"2024-09-16T04:49:16.953856+0000","last_undegraded":"2024-09-16T04:49:16.953856+0000","last_fullsized":"2024-09-16T04:49:16.953856+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2375,"log_dups_size":0,"ondisk_log_size":2375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:36.117906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9728000,"num_objects":2375,"num_object_clones":0,"num_object_copies":4750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2375,"num_write_kb":9500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2294","reported_seq":2346,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.426757+0000","last_change":"2024-09-16T04:49:08.115384+0000","last_active":"2024-09-16T04:49:16.426757+0000","last_peered":"2024-09-16T04:49:16.426757+0000","last_clean":"2024-09-16T04:49:16.426757+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:16.426757+0000","last_undegraded":"2024-09-16T04:49:16.426757+0000","last_fullsized":"2024-09-16T04:49:16.426757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2294,"log_dups_size":0,"ondisk_log_size":2294,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:19.310263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9372882,"num_objects":2290,"num_object_clones":0,"num_object_copies":4580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2292,"num_write_kb":9156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2291","reported_seq":2358,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.426620+0000","last_change":"2024-09-16T04:49:08.647868+0000","last_active":"2024-09-16T04:49:16.426620+0000","last_peered":"2024-09-16T04:49:16.426620+0000","last_clean":"2024-09-16T04:49:16.426620+0000","last_became_active":"2024-09-16T04:48:46.453489+0000","last_became_peered":"2024-09-16T04:48:46.453489+0000","last_unstale":"2024-09-16T04:49:16.426620+0000","last_undegraded":"2024-09-16T04:49:16.426620+0000","last_fullsized":"2024-09-16T04:49:16.426620+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2291,"log_dups_size":0,"ondisk_log_size":2291,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:20.282005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9368169,"num_objects":2288,"num_object_clones":0,"num_object_copies":4576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2290,"num_write_kb":9150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2297","reported_seq":2344,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.427024+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:16.427024+0000","last_peered":"2024-09-16T04:49:16.427024+0000","last_clean":"2024-09-16T04:49:16.427024+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:16.427024+0000","last_undegraded":"2024-09-16T04:49:16.427024+0000","last_fullsized":"2024-09-16T04:49:16.427024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2297,"log_dups_size":0,"ondisk_log_size":2297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9408512,"num_objects":2297,"num_object_clones":0,"num_object_copies":4594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2297,"num_write_kb":9188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2286","reported_seq":2344,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-16T04:49:16.971681+0000","last_change":"2024-09-16T04:48:46.451429+0000","last_active":"2024-09-16T04:49:16.971681+0000","last_peered":"2024-09-16T04:49:16.971681+0000","last_clean":"2024-09-16T04:48:09.846075+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:16.971681+0000","last_undegraded":"2024-09-16T04:49:16.971681+0000","last_fullsized":"2024-09-16T04:49:16.971681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2286,"log_dups_size":0,"ondisk_log_size":2286,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9336018,"num_objects":2281,"num_object_clones":0,"num_object_copies":4562,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2284,"num_write_kb":9120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2281}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2376","reported_seq":2430,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.973621+0000","last_change":"2024-09-16T04:48:45.442482+0000","last_active":"2024-09-16T04:49:16.973621+0000","last_peered":"2024-09-16T04:49:16.973621+0000","last_clean":"2024-09-16T04:48:11.202938+0000","last_became_active":"2024-09-16T04:48:45.441846+0000","last_became_peered":"2024-09-16T04:48:45.441846+0000","last_unstale":"2024-09-16T04:49:16.973621+0000","last_undegraded":"2024-09-16T04:48:45.421972+0000","last_fullsized":"2024-09-16T04:49:16.973621+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2376,"log_dups_size":0,"ondisk_log_size":2376,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9712850,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2374,"num_write_kb":9487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2373}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2356","reported_seq":2410,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.929375+0000","last_change":"2024-09-16T04:48:45.445456+0000","last_active":"2024-09-16T04:49:16.929375+0000","last_peered":"2024-09-16T04:49:16.929375+0000","last_clean":"2024-09-16T04:48:10.270497+0000","last_became_active":"2024-09-16T04:48:45.444639+0000","last_became_peered":"2024-09-16T04:48:45.444639+0000","last_unstale":"2024-09-16T04:49:16.929375+0000","last_undegraded":"2024-09-16T04:48:45.426169+0000","last_fullsized":"2024-09-16T04:49:16.929375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9626834,"num_objects":2352,"num_object_clones":0,"num_object_copies":4704,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2352}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2280","reported_seq":2340,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.975390+0000","last_change":"2024-09-16T04:48:45.440763+0000","last_active":"2024-09-16T04:49:16.975390+0000","last_peered":"2024-09-16T04:49:16.975390+0000","last_clean":"2024-09-16T04:47:58.315475+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:16.975390+0000","last_undegraded":"2024-09-16T04:48:45.420366+0000","last_fullsized":"2024-09-16T04:49:16.975390+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2280,"log_dups_size":0,"ondisk_log_size":2280,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9331305,"num_objects":2279,"num_object_clones":0,"num_object_copies":4558,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2279,"num_write_kb":9113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2279}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449263+0000","last_change":"2024-09-16T04:48:45.440924+0000","last_active":"2024-09-16T04:48:46.449263+0000","last_peered":"2024-09-16T04:48:46.449263+0000","last_clean":"2024-09-16T04:48:46.449263+0000","last_became_active":"2024-09-16T04:48:45.440123+0000","last_became_peered":"2024-09-16T04:48:45.440123+0000","last_unstale":"2024-09-16T04:48:46.449263+0000","last_undegraded":"2024-09-16T04:48:46.449263+0000","last_fullsized":"2024-09-16T04:48:46.449263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.338015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2258","reported_seq":2327,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.436485+0000","last_change":"2024-09-16T04:49:11.485127+0000","last_active":"2024-09-16T04:49:16.436485+0000","last_peered":"2024-09-16T04:49:16.436485+0000","last_clean":"2024-09-16T04:49:16.436485+0000","last_became_active":"2024-09-16T04:48:46.454004+0000","last_became_peered":"2024-09-16T04:48:46.454004+0000","last_unstale":"2024-09-16T04:49:16.436485+0000","last_undegraded":"2024-09-16T04:49:16.436485+0000","last_fullsized":"2024-09-16T04:49:16.436485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2258,"log_dups_size":0,"ondisk_log_size":2258,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:17.336710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9221330,"num_objects":2253,"num_object_clones":0,"num_object_copies":4506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":9008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.454453+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:48:46.454453+0000","last_peered":"2024-09-16T04:48:46.454453+0000","last_clean":"2024-09-16T04:48:46.454453+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:48:46.454453+0000","last_undegraded":"2024-09-16T04:48:46.454453+0000","last_fullsized":"2024-09-16T04:48:46.454453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2404","reported_seq":2453,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.929722+0000","last_change":"2024-09-16T04:48:45.445370+0000","last_active":"2024-09-16T04:49:16.929722+0000","last_peered":"2024-09-16T04:49:16.929722+0000","last_clean":"2024-09-16T04:48:11.292504+0000","last_became_active":"2024-09-16T04:48:45.444610+0000","last_became_peered":"2024-09-16T04:48:45.444610+0000","last_unstale":"2024-09-16T04:49:16.929722+0000","last_undegraded":"2024-09-16T04:48:45.422293+0000","last_fullsized":"2024-09-16T04:49:16.929722+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2404,"log_dups_size":0,"ondisk_log_size":2404,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9815250,"num_objects":2398,"num_object_clones":0,"num_object_copies":4796,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2402,"num_write_kb":9588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2398}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":61,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449124+0000","last_change":"2024-09-16T04:48:45.442094+0000","last_active":"2024-09-16T04:48:46.449124+0000","last_peered":"2024-09-16T04:48:46.449124+0000","last_clean":"2024-09-16T04:48:23.998001+0000","last_became_active":"2024-09-16T04:48:45.440291+0000","last_became_peered":"2024-09-16T04:48:45.440291+0000","last_unstale":"2024-09-16T04:48:46.449124+0000","last_undegraded":"2024-09-16T04:48:45.420971+0000","last_fullsized":"2024-09-16T04:48:46.449124+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2225","reported_seq":2284,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.416530+0000","last_change":"2024-09-16T04:49:12.621094+0000","last_active":"2024-09-16T04:49:16.416530+0000","last_peered":"2024-09-16T04:49:16.416530+0000","last_clean":"2024-09-16T04:49:16.416530+0000","last_became_active":"2024-09-16T04:48:46.452651+0000","last_became_peered":"2024-09-16T04:48:46.452651+0000","last_unstale":"2024-09-16T04:49:16.416530+0000","last_undegraded":"2024-09-16T04:49:16.416530+0000","last_fullsized":"2024-09-16T04:49:16.416530+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2225,"log_dups_size":0,"ondisk_log_size":2225,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:18.291240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9094354,"num_objects":2222,"num_object_clones":0,"num_object_copies":4444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2223,"num_write_kb":8883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.454753+0000","last_change":"2024-09-16T04:48:46.454753+0000","last_active":"2024-09-16T04:48:46.454753+0000","last_peered":"2024-09-16T04:48:46.454753+0000","last_clean":"2024-09-16T04:48:46.454753+0000","last_became_active":"2024-09-16T04:48:46.454400+0000","last_became_peered":"2024-09-16T04:48:46.454400+0000","last_unstale":"2024-09-16T04:48:46.454753+0000","last_undegraded":"2024-09-16T04:48:46.454753+0000","last_fullsized":"2024-09-16T04:48:46.454753+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:50:39.528170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2249","reported_seq":2308,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.420181+0000","last_change":"2024-09-16T04:49:13.167289+0000","last_active":"2024-09-16T04:49:16.420181+0000","last_peered":"2024-09-16T04:49:16.420181+0000","last_clean":"2024-09-16T04:49:16.420181+0000","last_became_active":"2024-09-16T04:48:46.454141+0000","last_became_peered":"2024-09-16T04:48:46.454141+0000","last_unstale":"2024-09-16T04:49:16.420181+0000","last_undegraded":"2024-09-16T04:49:16.420181+0000","last_fullsized":"2024-09-16T04:49:16.420181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2249,"log_dups_size":0,"ondisk_log_size":2249,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:18.291253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9188562,"num_objects":2245,"num_object_clones":0,"num_object_copies":4490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2247,"num_write_kb":8975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2357","reported_seq":2409,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.977239+0000","last_change":"2024-09-16T04:48:45.445513+0000","last_active":"2024-09-16T04:49:16.977239+0000","last_peered":"2024-09-16T04:49:16.977239+0000","last_clean":"2024-09-16T04:48:00.429368+0000","last_became_active":"2024-09-16T04:48:45.444748+0000","last_became_peered":"2024-09-16T04:48:45.444748+0000","last_unstale":"2024-09-16T04:49:16.977239+0000","last_undegraded":"2024-09-16T04:48:45.425112+0000","last_fullsized":"2024-09-16T04:49:16.977239+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2357,"log_dups_size":0,"ondisk_log_size":2357,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9646697,"num_objects":2356,"num_object_clones":0,"num_object_copies":4712,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2356,"num_write_kb":9421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2356}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449472+0000","last_change":"2024-09-16T04:48:45.443815+0000","last_active":"2024-09-16T04:48:46.449472+0000","last_peered":"2024-09-16T04:48:46.449472+0000","last_clean":"2024-09-16T04:48:46.449472+0000","last_became_active":"2024-09-16T04:48:45.443064+0000","last_became_peered":"2024-09-16T04:48:45.443064+0000","last_unstale":"2024-09-16T04:48:46.449472+0000","last_undegraded":"2024-09-16T04:48:46.449472+0000","last_fullsized":"2024-09-16T04:48:46.449472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:35.280081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2436","reported_seq":2489,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.968923+0000","last_change":"2024-09-16T04:48:45.446398+0000","last_active":"2024-09-16T04:49:16.968923+0000","last_peered":"2024-09-16T04:49:16.968923+0000","last_clean":"2024-09-16T04:48:11.121168+0000","last_became_active":"2024-09-16T04:48:45.445679+0000","last_became_peered":"2024-09-16T04:48:45.445679+0000","last_unstale":"2024-09-16T04:49:16.968923+0000","last_undegraded":"2024-09-16T04:48:45.426487+0000","last_fullsized":"2024-09-16T04:49:16.968923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2436,"log_dups_size":0,"ondisk_log_size":2436,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9946939,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2433,"num_write_kb":9716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2431}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.453526+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:48:46.453526+0000","last_peered":"2024-09-16T04:48:46.453526+0000","last_clean":"2024-09-16T04:48:46.453526+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:48:46.453526+0000","last_undegraded":"2024-09-16T04:48:46.453526+0000","last_fullsized":"2024-09-16T04:48:46.453526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2264","reported_seq":2317,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.968968+0000","last_change":"2024-09-16T04:48:45.444871+0000","last_active":"2024-09-16T04:49:16.968968+0000","last_peered":"2024-09-16T04:49:16.968968+0000","last_clean":"2024-09-16T04:48:00.428013+0000","last_became_active":"2024-09-16T04:48:45.444067+0000","last_became_peered":"2024-09-16T04:48:45.444067+0000","last_unstale":"2024-09-16T04:49:16.968968+0000","last_undegraded":"2024-09-16T04:48:45.426010+0000","last_fullsized":"2024-09-16T04:49:16.968968+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2264,"log_dups_size":0,"ondisk_log_size":2264,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9258194,"num_objects":2262,"num_object_clones":0,"num_object_copies":4524,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2262,"num_write_kb":9042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2262}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449269+0000","last_change":"2024-09-16T04:48:45.437264+0000","last_active":"2024-09-16T04:48:46.449269+0000","last_peered":"2024-09-16T04:48:46.449269+0000","last_clean":"2024-09-16T04:48:46.449269+0000","last_became_active":"2024-09-16T04:48:45.435646+0000","last_became_peered":"2024-09-16T04:48:45.435646+0000","last_unstale":"2024-09-16T04:48:46.449269+0000","last_undegraded":"2024-09-16T04:48:46.449269+0000","last_fullsized":"2024-09-16T04:48:46.449269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:29.272297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.453655+0000","last_change":"2024-09-16T04:48:46.453655+0000","last_active":"2024-09-16T04:48:46.453655+0000","last_peered":"2024-09-16T04:48:46.453655+0000","last_clean":"2024-09-16T04:48:46.453655+0000","last_became_active":"2024-09-16T04:48:46.453374+0000","last_became_peered":"2024-09-16T04:48:46.453374+0000","last_unstale":"2024-09-16T04:48:46.453655+0000","last_undegraded":"2024-09-16T04:48:46.453655+0000","last_fullsized":"2024-09-16T04:48:46.453655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:49:42.225940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2482","reported_seq":2536,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.947719+0000","last_change":"2024-09-16T04:48:45.445172+0000","last_active":"2024-09-16T04:49:16.947719+0000","last_peered":"2024-09-16T04:49:16.947719+0000","last_clean":"2024-09-16T04:48:10.711306+0000","last_became_active":"2024-09-16T04:48:45.440990+0000","last_became_peered":"2024-09-16T04:48:45.440990+0000","last_unstale":"2024-09-16T04:49:16.947719+0000","last_undegraded":"2024-09-16T04:48:45.422071+0000","last_fullsized":"2024-09-16T04:49:16.947719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10119588,"num_objects":2474,"num_object_clones":0,"num_object_copies":4948,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2479,"num_write_kb":9887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2473}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":152175484,"num_objects":37176,"num_object_clones":0,"num_object_copies":74352,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37203,"num_write_kb":148638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":303820800,"data_stored":303702514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37230,"ondisk_log_size":37230,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":258684,"kb_used_data":146476,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94113156,"statfs":{"total":96636764160,"available":96371871744,"internally_reserved":0,"allocated":149991424,"data_stored":149782848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":114880169},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182416,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":249272,"kb_used_data":145448,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94122568,"statfs":{"total":96636764160,"available":96381509632,"internally_reserved":0,"allocated":148938752,"data_stored":148705225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":106294954},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,0,2,8,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":153288704,"data_stored":153267830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":150417408,"data_stored":150417408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:19.531 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:20.049 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 23 20 41557 41537 7.0533 5.13281 0.0108659 0.0110672 2024-09-16T04:49:20.069 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:20.071 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:20.094 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-16T04:49:18.271757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":152175503,"num_objects":37178,"num_object_clones":0,"num_object_copies":74356,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37205,"num_write_kb":148640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37233,"ondisk_log_size":37233,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":507956,"kb_used_data":291924,"kb_used_omap":8,"kb_used_meta":215991,"kb_avail":188235724,"statfs":{"total":193273528320,"available":192753381376,"internally_reserved":0,"allocated":298930176,"data_stored":298488073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8877,"internal_metadata":221175123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,7,2,8,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":103456768,"num_objects":25258,"num_object_clones":0,"num_object_copies":50516,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":25258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25258,"num_write_kb":101032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002713"},"pg_stats":[{"pgid":"2.f","version":"36'2375","reported_seq":2422,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.953856+0000","last_change":"2024-09-16T04:48:45.444872+0000","last_active":"2024-09-16T04:49:16.953856+0000","last_peered":"2024-09-16T04:49:16.953856+0000","last_clean":"2024-09-16T04:49:16.953856+0000","last_became_active":"2024-09-16T04:48:45.444050+0000","last_became_peered":"2024-09-16T04:48:45.444050+0000","last_unstale":"2024-09-16T04:49:16.953856+0000","last_undegraded":"2024-09-16T04:49:16.953856+0000","last_fullsized":"2024-09-16T04:49:16.953856+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2375,"log_dups_size":0,"ondisk_log_size":2375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:36.117906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9728000,"num_objects":2375,"num_object_clones":0,"num_object_copies":4750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2375,"num_write_kb":9500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2294","reported_seq":2346,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.426757+0000","last_change":"2024-09-16T04:49:08.115384+0000","last_active":"2024-09-16T04:49:16.426757+0000","last_peered":"2024-09-16T04:49:16.426757+0000","last_clean":"2024-09-16T04:49:16.426757+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:16.426757+0000","last_undegraded":"2024-09-16T04:49:16.426757+0000","last_fullsized":"2024-09-16T04:49:16.426757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2294,"log_dups_size":0,"ondisk_log_size":2294,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:19.310263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9372882,"num_objects":2290,"num_object_clones":0,"num_object_copies":4580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2292,"num_write_kb":9156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2291","reported_seq":2358,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.426620+0000","last_change":"2024-09-16T04:49:08.647868+0000","last_active":"2024-09-16T04:49:16.426620+0000","last_peered":"2024-09-16T04:49:16.426620+0000","last_clean":"2024-09-16T04:49:16.426620+0000","last_became_active":"2024-09-16T04:48:46.453489+0000","last_became_peered":"2024-09-16T04:48:46.453489+0000","last_unstale":"2024-09-16T04:49:16.426620+0000","last_undegraded":"2024-09-16T04:49:16.426620+0000","last_fullsized":"2024-09-16T04:49:16.426620+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2291,"log_dups_size":0,"ondisk_log_size":2291,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:20.282005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9368169,"num_objects":2288,"num_object_clones":0,"num_object_copies":4576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2290,"num_write_kb":9150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2297","reported_seq":2344,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.427024+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:16.427024+0000","last_peered":"2024-09-16T04:49:16.427024+0000","last_clean":"2024-09-16T04:49:16.427024+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:16.427024+0000","last_undegraded":"2024-09-16T04:49:16.427024+0000","last_fullsized":"2024-09-16T04:49:16.427024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2297,"log_dups_size":0,"ondisk_log_size":2297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9408512,"num_objects":2297,"num_object_clones":0,"num_object_copies":4594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2297,"num_write_kb":9188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2286","reported_seq":2344,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-16T04:49:16.971681+0000","last_change":"2024-09-16T04:48:46.451429+0000","last_active":"2024-09-16T04:49:16.971681+0000","last_peered":"2024-09-16T04:49:16.971681+0000","last_clean":"2024-09-16T04:48:09.846075+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:16.971681+0000","last_undegraded":"2024-09-16T04:49:16.971681+0000","last_fullsized":"2024-09-16T04:49:16.971681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2286,"log_dups_size":0,"ondisk_log_size":2286,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9336018,"num_objects":2281,"num_object_clones":0,"num_object_copies":4562,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2284,"num_write_kb":9120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2281}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2376","reported_seq":2430,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.973621+0000","last_change":"2024-09-16T04:48:45.442482+0000","last_active":"2024-09-16T04:49:16.973621+0000","last_peered":"2024-09-16T04:49:16.973621+0000","last_clean":"2024-09-16T04:48:11.202938+0000","last_became_active":"2024-09-16T04:48:45.441846+0000","last_became_peered":"2024-09-16T04:48:45.441846+0000","last_unstale":"2024-09-16T04:49:16.973621+0000","last_undegraded":"2024-09-16T04:48:45.421972+0000","last_fullsized":"2024-09-16T04:49:16.973621+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2376,"log_dups_size":0,"ondisk_log_size":2376,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9712850,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2374,"num_write_kb":9487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2373}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2356","reported_seq":2410,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.929375+0000","last_change":"2024-09-16T04:48:45.445456+0000","last_active":"2024-09-16T04:49:16.929375+0000","last_peered":"2024-09-16T04:49:16.929375+0000","last_clean":"2024-09-16T04:48:10.270497+0000","last_became_active":"2024-09-16T04:48:45.444639+0000","last_became_peered":"2024-09-16T04:48:45.444639+0000","last_unstale":"2024-09-16T04:49:16.929375+0000","last_undegraded":"2024-09-16T04:48:45.426169+0000","last_fullsized":"2024-09-16T04:49:16.929375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9626834,"num_objects":2352,"num_object_clones":0,"num_object_copies":4704,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2352}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2280","reported_seq":2340,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.975390+0000","last_change":"2024-09-16T04:48:45.440763+0000","last_active":"2024-09-16T04:49:16.975390+0000","last_peered":"2024-09-16T04:49:16.975390+0000","last_clean":"2024-09-16T04:47:58.315475+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:16.975390+0000","last_undegraded":"2024-09-16T04:48:45.420366+0000","last_fullsized":"2024-09-16T04:49:16.975390+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2280,"log_dups_size":0,"ondisk_log_size":2280,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9331305,"num_objects":2279,"num_object_clones":0,"num_object_copies":4558,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2279,"num_write_kb":9113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2279}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449263+0000","last_change":"2024-09-16T04:48:45.440924+0000","last_active":"2024-09-16T04:48:46.449263+0000","last_peered":"2024-09-16T04:48:46.449263+0000","last_clean":"2024-09-16T04:48:46.449263+0000","last_became_active":"2024-09-16T04:48:45.440123+0000","last_became_peered":"2024-09-16T04:48:45.440123+0000","last_unstale":"2024-09-16T04:48:46.449263+0000","last_undegraded":"2024-09-16T04:48:46.449263+0000","last_fullsized":"2024-09-16T04:48:46.449263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.338015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2258","reported_seq":2327,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.436485+0000","last_change":"2024-09-16T04:49:11.485127+0000","last_active":"2024-09-16T04:49:16.436485+0000","last_peered":"2024-09-16T04:49:16.436485+0000","last_clean":"2024-09-16T04:49:16.436485+0000","last_became_active":"2024-09-16T04:48:46.454004+0000","last_became_peered":"2024-09-16T04:48:46.454004+0000","last_unstale":"2024-09-16T04:49:16.436485+0000","last_undegraded":"2024-09-16T04:49:16.436485+0000","last_fullsized":"2024-09-16T04:49:16.436485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2258,"log_dups_size":0,"ondisk_log_size":2258,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:17.336710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9221330,"num_objects":2253,"num_object_clones":0,"num_object_copies":4506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":9008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.454453+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:48:46.454453+0000","last_peered":"2024-09-16T04:48:46.454453+0000","last_clean":"2024-09-16T04:48:46.454453+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:48:46.454453+0000","last_undegraded":"2024-09-16T04:48:46.454453+0000","last_fullsized":"2024-09-16T04:48:46.454453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2404","reported_seq":2453,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.929722+0000","last_change":"2024-09-16T04:48:45.445370+0000","last_active":"2024-09-16T04:49:16.929722+0000","last_peered":"2024-09-16T04:49:16.929722+0000","last_clean":"2024-09-16T04:48:11.292504+0000","last_became_active":"2024-09-16T04:48:45.444610+0000","last_became_peered":"2024-09-16T04:48:45.444610+0000","last_unstale":"2024-09-16T04:49:16.929722+0000","last_undegraded":"2024-09-16T04:48:45.422293+0000","last_fullsized":"2024-09-16T04:49:16.929722+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2404,"log_dups_size":0,"ondisk_log_size":2404,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9815250,"num_objects":2398,"num_object_clones":0,"num_object_copies":4796,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2402,"num_write_kb":9588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2398}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":61,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449124+0000","last_change":"2024-09-16T04:48:45.442094+0000","last_active":"2024-09-16T04:48:46.449124+0000","last_peered":"2024-09-16T04:48:46.449124+0000","last_clean":"2024-09-16T04:48:23.998001+0000","last_became_active":"2024-09-16T04:48:45.440291+0000","last_became_peered":"2024-09-16T04:48:45.440291+0000","last_unstale":"2024-09-16T04:48:46.449124+0000","last_undegraded":"2024-09-16T04:48:45.420971+0000","last_fullsized":"2024-09-16T04:48:46.449124+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2225","reported_seq":2284,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.416530+0000","last_change":"2024-09-16T04:49:12.621094+0000","last_active":"2024-09-16T04:49:16.416530+0000","last_peered":"2024-09-16T04:49:16.416530+0000","last_clean":"2024-09-16T04:49:16.416530+0000","last_became_active":"2024-09-16T04:48:46.452651+0000","last_became_peered":"2024-09-16T04:48:46.452651+0000","last_unstale":"2024-09-16T04:49:16.416530+0000","last_undegraded":"2024-09-16T04:49:16.416530+0000","last_fullsized":"2024-09-16T04:49:16.416530+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2225,"log_dups_size":0,"ondisk_log_size":2225,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:18.291240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9094354,"num_objects":2222,"num_object_clones":0,"num_object_copies":4444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2223,"num_write_kb":8883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.454753+0000","last_change":"2024-09-16T04:48:46.454753+0000","last_active":"2024-09-16T04:48:46.454753+0000","last_peered":"2024-09-16T04:48:46.454753+0000","last_clean":"2024-09-16T04:48:46.454753+0000","last_became_active":"2024-09-16T04:48:46.454400+0000","last_became_peered":"2024-09-16T04:48:46.454400+0000","last_unstale":"2024-09-16T04:48:46.454753+0000","last_undegraded":"2024-09-16T04:48:46.454753+0000","last_fullsized":"2024-09-16T04:48:46.454753+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:50:39.528170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2249","reported_seq":2308,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.420181+0000","last_change":"2024-09-16T04:49:13.167289+0000","last_active":"2024-09-16T04:49:16.420181+0000","last_peered":"2024-09-16T04:49:16.420181+0000","last_clean":"2024-09-16T04:49:16.420181+0000","last_became_active":"2024-09-16T04:48:46.454141+0000","last_became_peered":"2024-09-16T04:48:46.454141+0000","last_unstale":"2024-09-16T04:49:16.420181+0000","last_undegraded":"2024-09-16T04:49:16.420181+0000","last_fullsized":"2024-09-16T04:49:16.420181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2249,"log_dups_size":0,"ondisk_log_size":2249,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:18.291253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9188562,"num_objects":2245,"num_object_clones":0,"num_object_copies":4490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2247,"num_write_kb":8975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2357","reported_seq":2409,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.977239+0000","last_change":"2024-09-16T04:48:45.445513+0000","last_active":"2024-09-16T04:49:16.977239+0000","last_peered":"2024-09-16T04:49:16.977239+0000","last_clean":"2024-09-16T04:48:00.429368+0000","last_became_active":"2024-09-16T04:48:45.444748+0000","last_became_peered":"2024-09-16T04:48:45.444748+0000","last_unstale":"2024-09-16T04:49:16.977239+0000","last_undegraded":"2024-09-16T04:48:45.425112+0000","last_fullsized":"2024-09-16T04:49:16.977239+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2357,"log_dups_size":0,"ondisk_log_size":2357,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9646697,"num_objects":2356,"num_object_clones":0,"num_object_copies":4712,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2356,"num_write_kb":9421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2356}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449472+0000","last_change":"2024-09-16T04:48:45.443815+0000","last_active":"2024-09-16T04:48:46.449472+0000","last_peered":"2024-09-16T04:48:46.449472+0000","last_clean":"2024-09-16T04:48:46.449472+0000","last_became_active":"2024-09-16T04:48:45.443064+0000","last_became_peered":"2024-09-16T04:48:45.443064+0000","last_unstale":"2024-09-16T04:48:46.449472+0000","last_undegraded":"2024-09-16T04:48:46.449472+0000","last_fullsized":"2024-09-16T04:48:46.449472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:35.280081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2436","reported_seq":2489,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.968923+0000","last_change":"2024-09-16T04:48:45.446398+0000","last_active":"2024-09-16T04:49:16.968923+0000","last_peered":"2024-09-16T04:49:16.968923+0000","last_clean":"2024-09-16T04:48:11.121168+0000","last_became_active":"2024-09-16T04:48:45.445679+0000","last_became_peered":"2024-09-16T04:48:45.445679+0000","last_unstale":"2024-09-16T04:49:16.968923+0000","last_undegraded":"2024-09-16T04:48:45.426487+0000","last_fullsized":"2024-09-16T04:49:16.968923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2436,"log_dups_size":0,"ondisk_log_size":2436,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9946939,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2433,"num_write_kb":9716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2431}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.453526+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:48:46.453526+0000","last_peered":"2024-09-16T04:48:46.453526+0000","last_clean":"2024-09-16T04:48:46.453526+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:48:46.453526+0000","last_undegraded":"2024-09-16T04:48:46.453526+0000","last_fullsized":"2024-09-16T04:48:46.453526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2264","reported_seq":2317,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.968968+0000","last_change":"2024-09-16T04:48:45.444871+0000","last_active":"2024-09-16T04:49:16.968968+0000","last_peered":"2024-09-16T04:49:16.968968+0000","last_clean":"2024-09-16T04:48:00.428013+0000","last_became_active":"2024-09-16T04:48:45.444067+0000","last_became_peered":"2024-09-16T04:48:45.444067+0000","last_unstale":"2024-09-16T04:49:16.968968+0000","last_undegraded":"2024-09-16T04:48:45.426010+0000","last_fullsized":"2024-09-16T04:49:16.968968+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2264,"log_dups_size":0,"ondisk_log_size":2264,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9258194,"num_objects":2262,"num_object_clones":0,"num_object_copies":4524,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2262,"num_write_kb":9042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2262}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449269+0000","last_change":"2024-09-16T04:48:45.437264+0000","last_active":"2024-09-16T04:48:46.449269+0000","last_peered":"2024-09-16T04:48:46.449269+0000","last_clean":"2024-09-16T04:48:46.449269+0000","last_became_active":"2024-09-16T04:48:45.435646+0000","last_became_peered":"2024-09-16T04:48:45.435646+0000","last_unstale":"2024-09-16T04:48:46.449269+0000","last_undegraded":"2024-09-16T04:48:46.449269+0000","last_fullsized":"2024-09-16T04:48:46.449269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:29.272297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.453655+0000","last_change":"2024-09-16T04:48:46.453655+0000","last_active":"2024-09-16T04:48:46.453655+0000","last_peered":"2024-09-16T04:48:46.453655+0000","last_clean":"2024-09-16T04:48:46.453655+0000","last_became_active":"2024-09-16T04:48:46.453374+0000","last_became_peered":"2024-09-16T04:48:46.453374+0000","last_unstale":"2024-09-16T04:48:46.453655+0000","last_undegraded":"2024-09-16T04:48:46.453655+0000","last_fullsized":"2024-09-16T04:48:46.453655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:49:42.225940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2482","reported_seq":2536,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.947719+0000","last_change":"2024-09-16T04:48:45.445172+0000","last_active":"2024-09-16T04:49:16.947719+0000","last_peered":"2024-09-16T04:49:16.947719+0000","last_clean":"2024-09-16T04:48:10.711306+0000","last_became_active":"2024-09-16T04:48:45.440990+0000","last_became_peered":"2024-09-16T04:48:45.440990+0000","last_unstale":"2024-09-16T04:49:16.947719+0000","last_undegraded":"2024-09-16T04:48:45.422071+0000","last_fullsized":"2024-09-16T04:49:16.947719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10119588,"num_objects":2474,"num_object_clones":0,"num_object_copies":4948,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2479,"num_write_kb":9887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2473}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":152175484,"num_objects":37176,"num_object_clones":0,"num_object_copies":74352,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37203,"num_write_kb":148638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":303820800,"data_stored":303702514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37230,"ondisk_log_size":37230,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":258684,"kb_used_data":146476,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94113156,"statfs":{"total":96636764160,"available":96371871744,"internally_reserved":0,"allocated":149991424,"data_stored":149782848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":114880169},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182416,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":249272,"kb_used_data":145448,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94122568,"statfs":{"total":96636764160,"available":96381509632,"internally_reserved":0,"allocated":148938752,"data_stored":148705225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":106294954},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,0,2,8,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":153288704,"data_stored":153267830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":150417408,"data_stored":150417408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:20.096 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:20.716 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:20.717 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:20.749 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-09-16T04:49:20.272218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":152175503,"num_objects":37178,"num_object_clones":0,"num_object_copies":74356,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37205,"num_write_kb":148640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37233,"ondisk_log_size":37233,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":507956,"kb_used_data":291924,"kb_used_omap":8,"kb_used_meta":215991,"kb_avail":188235724,"statfs":{"total":193273528320,"available":192753381376,"internally_reserved":0,"allocated":298930176,"data_stored":298488073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8877,"internal_metadata":221175123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,7,2,8,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":60485632,"num_objects":14767,"num_object_clones":0,"num_object_copies":29534,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":14767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14767,"num_write_kb":59068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002730"},"pg_stats":[{"pgid":"2.f","version":"36'2375","reported_seq":2422,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.953856+0000","last_change":"2024-09-16T04:48:45.444872+0000","last_active":"2024-09-16T04:49:16.953856+0000","last_peered":"2024-09-16T04:49:16.953856+0000","last_clean":"2024-09-16T04:49:16.953856+0000","last_became_active":"2024-09-16T04:48:45.444050+0000","last_became_peered":"2024-09-16T04:48:45.444050+0000","last_unstale":"2024-09-16T04:49:16.953856+0000","last_undegraded":"2024-09-16T04:49:16.953856+0000","last_fullsized":"2024-09-16T04:49:16.953856+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2375,"log_dups_size":0,"ondisk_log_size":2375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:36.117906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9728000,"num_objects":2375,"num_object_clones":0,"num_object_copies":4750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2375,"num_write_kb":9500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2294","reported_seq":2346,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.426757+0000","last_change":"2024-09-16T04:49:08.115384+0000","last_active":"2024-09-16T04:49:16.426757+0000","last_peered":"2024-09-16T04:49:16.426757+0000","last_clean":"2024-09-16T04:49:16.426757+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:16.426757+0000","last_undegraded":"2024-09-16T04:49:16.426757+0000","last_fullsized":"2024-09-16T04:49:16.426757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2294,"log_dups_size":0,"ondisk_log_size":2294,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:19.310263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9372882,"num_objects":2290,"num_object_clones":0,"num_object_copies":4580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2292,"num_write_kb":9156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2291","reported_seq":2358,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.426620+0000","last_change":"2024-09-16T04:49:08.647868+0000","last_active":"2024-09-16T04:49:16.426620+0000","last_peered":"2024-09-16T04:49:16.426620+0000","last_clean":"2024-09-16T04:49:16.426620+0000","last_became_active":"2024-09-16T04:48:46.453489+0000","last_became_peered":"2024-09-16T04:48:46.453489+0000","last_unstale":"2024-09-16T04:49:16.426620+0000","last_undegraded":"2024-09-16T04:49:16.426620+0000","last_fullsized":"2024-09-16T04:49:16.426620+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2291,"log_dups_size":0,"ondisk_log_size":2291,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:20.282005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9368169,"num_objects":2288,"num_object_clones":0,"num_object_copies":4576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2290,"num_write_kb":9150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2297","reported_seq":2344,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.427024+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:16.427024+0000","last_peered":"2024-09-16T04:49:16.427024+0000","last_clean":"2024-09-16T04:49:16.427024+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:16.427024+0000","last_undegraded":"2024-09-16T04:49:16.427024+0000","last_fullsized":"2024-09-16T04:49:16.427024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2297,"log_dups_size":0,"ondisk_log_size":2297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9408512,"num_objects":2297,"num_object_clones":0,"num_object_copies":4594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2297,"num_write_kb":9188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2286","reported_seq":2344,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-16T04:49:16.971681+0000","last_change":"2024-09-16T04:48:46.451429+0000","last_active":"2024-09-16T04:49:16.971681+0000","last_peered":"2024-09-16T04:49:16.971681+0000","last_clean":"2024-09-16T04:48:09.846075+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:16.971681+0000","last_undegraded":"2024-09-16T04:49:16.971681+0000","last_fullsized":"2024-09-16T04:49:16.971681+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2286,"log_dups_size":0,"ondisk_log_size":2286,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9336018,"num_objects":2281,"num_object_clones":0,"num_object_copies":4562,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2284,"num_write_kb":9120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2281}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2376","reported_seq":2430,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.973621+0000","last_change":"2024-09-16T04:48:45.442482+0000","last_active":"2024-09-16T04:49:16.973621+0000","last_peered":"2024-09-16T04:49:16.973621+0000","last_clean":"2024-09-16T04:48:11.202938+0000","last_became_active":"2024-09-16T04:48:45.441846+0000","last_became_peered":"2024-09-16T04:48:45.441846+0000","last_unstale":"2024-09-16T04:49:16.973621+0000","last_undegraded":"2024-09-16T04:48:45.421972+0000","last_fullsized":"2024-09-16T04:49:16.973621+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2376,"log_dups_size":0,"ondisk_log_size":2376,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9712850,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2374,"num_write_kb":9487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2373}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2356","reported_seq":2410,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.929375+0000","last_change":"2024-09-16T04:48:45.445456+0000","last_active":"2024-09-16T04:49:16.929375+0000","last_peered":"2024-09-16T04:49:16.929375+0000","last_clean":"2024-09-16T04:48:10.270497+0000","last_became_active":"2024-09-16T04:48:45.444639+0000","last_became_peered":"2024-09-16T04:48:45.444639+0000","last_unstale":"2024-09-16T04:49:16.929375+0000","last_undegraded":"2024-09-16T04:48:45.426169+0000","last_fullsized":"2024-09-16T04:49:16.929375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9626834,"num_objects":2352,"num_object_clones":0,"num_object_copies":4704,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2354,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2352}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2280","reported_seq":2340,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.975390+0000","last_change":"2024-09-16T04:48:45.440763+0000","last_active":"2024-09-16T04:49:16.975390+0000","last_peered":"2024-09-16T04:49:16.975390+0000","last_clean":"2024-09-16T04:47:58.315475+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:16.975390+0000","last_undegraded":"2024-09-16T04:48:45.420366+0000","last_fullsized":"2024-09-16T04:49:16.975390+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2280,"log_dups_size":0,"ondisk_log_size":2280,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9331305,"num_objects":2279,"num_object_clones":0,"num_object_copies":4558,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2279,"num_write_kb":9113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2279}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449263+0000","last_change":"2024-09-16T04:48:45.440924+0000","last_active":"2024-09-16T04:48:46.449263+0000","last_peered":"2024-09-16T04:48:46.449263+0000","last_clean":"2024-09-16T04:48:46.449263+0000","last_became_active":"2024-09-16T04:48:45.440123+0000","last_became_peered":"2024-09-16T04:48:45.440123+0000","last_unstale":"2024-09-16T04:48:46.449263+0000","last_undegraded":"2024-09-16T04:48:46.449263+0000","last_fullsized":"2024-09-16T04:48:46.449263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.338015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2258","reported_seq":2327,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.436485+0000","last_change":"2024-09-16T04:49:11.485127+0000","last_active":"2024-09-16T04:49:16.436485+0000","last_peered":"2024-09-16T04:49:16.436485+0000","last_clean":"2024-09-16T04:49:16.436485+0000","last_became_active":"2024-09-16T04:48:46.454004+0000","last_became_peered":"2024-09-16T04:48:46.454004+0000","last_unstale":"2024-09-16T04:49:16.436485+0000","last_undegraded":"2024-09-16T04:49:16.436485+0000","last_fullsized":"2024-09-16T04:49:16.436485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2258,"log_dups_size":0,"ondisk_log_size":2258,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:17.336710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9221330,"num_objects":2253,"num_object_clones":0,"num_object_copies":4506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2256,"num_write_kb":9008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.454453+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:48:46.454453+0000","last_peered":"2024-09-16T04:48:46.454453+0000","last_clean":"2024-09-16T04:48:46.454453+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:48:46.454453+0000","last_undegraded":"2024-09-16T04:48:46.454453+0000","last_fullsized":"2024-09-16T04:48:46.454453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2404","reported_seq":2453,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.929722+0000","last_change":"2024-09-16T04:48:45.445370+0000","last_active":"2024-09-16T04:49:16.929722+0000","last_peered":"2024-09-16T04:49:16.929722+0000","last_clean":"2024-09-16T04:48:11.292504+0000","last_became_active":"2024-09-16T04:48:45.444610+0000","last_became_peered":"2024-09-16T04:48:45.444610+0000","last_unstale":"2024-09-16T04:49:16.929722+0000","last_undegraded":"2024-09-16T04:48:45.422293+0000","last_fullsized":"2024-09-16T04:49:16.929722+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2404,"log_dups_size":0,"ondisk_log_size":2404,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9815250,"num_objects":2398,"num_object_clones":0,"num_object_copies":4796,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2402,"num_write_kb":9588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2398}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":61,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:48:46.449124+0000","last_change":"2024-09-16T04:48:45.442094+0000","last_active":"2024-09-16T04:48:46.449124+0000","last_peered":"2024-09-16T04:48:46.449124+0000","last_clean":"2024-09-16T04:48:23.998001+0000","last_became_active":"2024-09-16T04:48:45.440291+0000","last_became_peered":"2024-09-16T04:48:45.440291+0000","last_unstale":"2024-09-16T04:48:46.449124+0000","last_undegraded":"2024-09-16T04:48:45.420971+0000","last_fullsized":"2024-09-16T04:48:46.449124+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2225","reported_seq":2284,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.416530+0000","last_change":"2024-09-16T04:49:12.621094+0000","last_active":"2024-09-16T04:49:16.416530+0000","last_peered":"2024-09-16T04:49:16.416530+0000","last_clean":"2024-09-16T04:49:16.416530+0000","last_became_active":"2024-09-16T04:48:46.452651+0000","last_became_peered":"2024-09-16T04:48:46.452651+0000","last_unstale":"2024-09-16T04:49:16.416530+0000","last_undegraded":"2024-09-16T04:49:16.416530+0000","last_fullsized":"2024-09-16T04:49:16.416530+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2225,"log_dups_size":0,"ondisk_log_size":2225,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:18.291240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9094354,"num_objects":2222,"num_object_clones":0,"num_object_copies":4444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2223,"num_write_kb":8883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.454753+0000","last_change":"2024-09-16T04:48:46.454753+0000","last_active":"2024-09-16T04:48:46.454753+0000","last_peered":"2024-09-16T04:48:46.454753+0000","last_clean":"2024-09-16T04:48:46.454753+0000","last_became_active":"2024-09-16T04:48:46.454400+0000","last_became_peered":"2024-09-16T04:48:46.454400+0000","last_unstale":"2024-09-16T04:48:46.454753+0000","last_undegraded":"2024-09-16T04:48:46.454753+0000","last_fullsized":"2024-09-16T04:48:46.454753+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:50:39.528170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2249","reported_seq":2308,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:16.420181+0000","last_change":"2024-09-16T04:49:13.167289+0000","last_active":"2024-09-16T04:49:16.420181+0000","last_peered":"2024-09-16T04:49:16.420181+0000","last_clean":"2024-09-16T04:49:16.420181+0000","last_became_active":"2024-09-16T04:48:46.454141+0000","last_became_peered":"2024-09-16T04:48:46.454141+0000","last_unstale":"2024-09-16T04:49:16.420181+0000","last_undegraded":"2024-09-16T04:49:16.420181+0000","last_fullsized":"2024-09-16T04:49:16.420181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2249,"log_dups_size":0,"ondisk_log_size":2249,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:18.291253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9188562,"num_objects":2245,"num_object_clones":0,"num_object_copies":4490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2247,"num_write_kb":8975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2357","reported_seq":2409,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.977239+0000","last_change":"2024-09-16T04:48:45.445513+0000","last_active":"2024-09-16T04:49:16.977239+0000","last_peered":"2024-09-16T04:49:16.977239+0000","last_clean":"2024-09-16T04:48:00.429368+0000","last_became_active":"2024-09-16T04:48:45.444748+0000","last_became_peered":"2024-09-16T04:48:45.444748+0000","last_unstale":"2024-09-16T04:49:16.977239+0000","last_undegraded":"2024-09-16T04:48:45.425112+0000","last_fullsized":"2024-09-16T04:49:16.977239+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2357,"log_dups_size":0,"ondisk_log_size":2357,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9646697,"num_objects":2356,"num_object_clones":0,"num_object_copies":4712,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2356,"num_write_kb":9421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2356}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449472+0000","last_change":"2024-09-16T04:48:45.443815+0000","last_active":"2024-09-16T04:48:46.449472+0000","last_peered":"2024-09-16T04:48:46.449472+0000","last_clean":"2024-09-16T04:48:46.449472+0000","last_became_active":"2024-09-16T04:48:45.443064+0000","last_became_peered":"2024-09-16T04:48:45.443064+0000","last_unstale":"2024-09-16T04:48:46.449472+0000","last_undegraded":"2024-09-16T04:48:46.449472+0000","last_fullsized":"2024-09-16T04:48:46.449472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:35.280081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2436","reported_seq":2489,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.968923+0000","last_change":"2024-09-16T04:48:45.446398+0000","last_active":"2024-09-16T04:49:16.968923+0000","last_peered":"2024-09-16T04:49:16.968923+0000","last_clean":"2024-09-16T04:48:11.121168+0000","last_became_active":"2024-09-16T04:48:45.445679+0000","last_became_peered":"2024-09-16T04:48:45.445679+0000","last_unstale":"2024-09-16T04:49:16.968923+0000","last_undegraded":"2024-09-16T04:48:45.426487+0000","last_fullsized":"2024-09-16T04:49:16.968923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2436,"log_dups_size":0,"ondisk_log_size":2436,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9946939,"num_objects":2431,"num_object_clones":0,"num_object_copies":4862,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2433,"num_write_kb":9716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2431}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.453526+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:48:46.453526+0000","last_peered":"2024-09-16T04:48:46.453526+0000","last_clean":"2024-09-16T04:48:46.453526+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:48:46.453526+0000","last_undegraded":"2024-09-16T04:48:46.453526+0000","last_fullsized":"2024-09-16T04:48:46.453526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2264","reported_seq":2317,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.968968+0000","last_change":"2024-09-16T04:48:45.444871+0000","last_active":"2024-09-16T04:49:16.968968+0000","last_peered":"2024-09-16T04:49:16.968968+0000","last_clean":"2024-09-16T04:48:00.428013+0000","last_became_active":"2024-09-16T04:48:45.444067+0000","last_became_peered":"2024-09-16T04:48:45.444067+0000","last_unstale":"2024-09-16T04:49:16.968968+0000","last_undegraded":"2024-09-16T04:48:45.426010+0000","last_fullsized":"2024-09-16T04:49:16.968968+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2264,"log_dups_size":0,"ondisk_log_size":2264,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9258194,"num_objects":2262,"num_object_clones":0,"num_object_copies":4524,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2262,"num_write_kb":9042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2262}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449269+0000","last_change":"2024-09-16T04:48:45.437264+0000","last_active":"2024-09-16T04:48:46.449269+0000","last_peered":"2024-09-16T04:48:46.449269+0000","last_clean":"2024-09-16T04:48:46.449269+0000","last_became_active":"2024-09-16T04:48:45.435646+0000","last_became_peered":"2024-09-16T04:48:45.435646+0000","last_unstale":"2024-09-16T04:48:46.449269+0000","last_undegraded":"2024-09-16T04:48:46.449269+0000","last_fullsized":"2024-09-16T04:48:46.449269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:29.272297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.453655+0000","last_change":"2024-09-16T04:48:46.453655+0000","last_active":"2024-09-16T04:48:46.453655+0000","last_peered":"2024-09-16T04:48:46.453655+0000","last_clean":"2024-09-16T04:48:46.453655+0000","last_became_active":"2024-09-16T04:48:46.453374+0000","last_became_peered":"2024-09-16T04:48:46.453374+0000","last_unstale":"2024-09-16T04:48:46.453655+0000","last_undegraded":"2024-09-16T04:48:46.453655+0000","last_fullsized":"2024-09-16T04:48:46.453655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:49:42.225940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2482","reported_seq":2536,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T04:49:16.947719+0000","last_change":"2024-09-16T04:48:45.445172+0000","last_active":"2024-09-16T04:49:16.947719+0000","last_peered":"2024-09-16T04:49:16.947719+0000","last_clean":"2024-09-16T04:48:10.711306+0000","last_became_active":"2024-09-16T04:48:45.440990+0000","last_became_peered":"2024-09-16T04:48:45.440990+0000","last_unstale":"2024-09-16T04:49:16.947719+0000","last_undegraded":"2024-09-16T04:48:45.422071+0000","last_fullsized":"2024-09-16T04:49:16.947719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10119588,"num_objects":2474,"num_object_clones":0,"num_object_copies":4948,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2479,"num_write_kb":9887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2473}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":152175484,"num_objects":37176,"num_object_clones":0,"num_object_copies":74352,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37203,"num_write_kb":148638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":303820800,"data_stored":303702514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37230,"ondisk_log_size":37230,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":258684,"kb_used_data":146476,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94113156,"statfs":{"total":96636764160,"available":96371871744,"internally_reserved":0,"allocated":149991424,"data_stored":149782848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":114880169},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182416,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":249272,"kb_used_data":145448,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94122568,"statfs":{"total":96636764160,"available":96381509632,"internally_reserved":0,"allocated":148938752,"data_stored":148705225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":106294954},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,0,2,8,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":153288704,"data_stored":153267830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":150417408,"data_stored":150417408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:21.050 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 24 20 42880 42860 6.97455 5.16797 0.0106197 0.0111951 2024-09-16T04:49:22.050 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 25 20 44304 44284 6.91806 5.5625 0.0119962 0.0112868 2024-09-16T04:49:23.050 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 26 20 45583 45563 6.84413 4.99609 0.00513312 0.011403 2024-09-16T04:49:23.752 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:24.050 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 27 20 46755 46735 6.7602 4.57812 0.0156298 0.0115463 2024-09-16T04:49:24.425 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:24.426 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:24.465 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-09-16T04:49:24.273000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":179929999,"num_objects":43954,"num_object_clones":0,"num_object_copies":87908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43981,"num_write_kb":175744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":636,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44009,"ondisk_log_size":44009,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":565804,"kb_used_data":345612,"kb_used_omap":8,"kb_used_meta":220151,"kb_avail":188177876,"statfs":{"total":193273528320,"available":192694145024,"internally_reserved":0,"allocated":353906688,"data_stored":353464585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8877,"internal_metadata":225434963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,4,8,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":56283136,"num_objects":13741,"num_object_clones":0,"num_object_copies":27482,"num_objects_missing_on_primary":-24,"num_objects_missing":-24,"num_objects_degraded":-42,"num_objects_misplaced":0,"num_objects_unfound":-13,"num_objects_dirty":13741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13741,"num_write_kb":54964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":636,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002588"},"pg_stats":[{"pgid":"2.f","version":"36'2756","reported_seq":2803,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.956227+0000","last_change":"2024-09-16T04:48:45.444872+0000","last_active":"2024-09-16T04:49:21.956227+0000","last_peered":"2024-09-16T04:49:21.956227+0000","last_clean":"2024-09-16T04:49:21.956227+0000","last_became_active":"2024-09-16T04:48:45.444050+0000","last_became_peered":"2024-09-16T04:48:45.444050+0000","last_unstale":"2024-09-16T04:49:21.956227+0000","last_undegraded":"2024-09-16T04:49:21.956227+0000","last_fullsized":"2024-09-16T04:49:21.956227+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2756,"log_dups_size":0,"ondisk_log_size":2756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:36.117906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11288576,"num_objects":2756,"num_object_clones":0,"num_object_copies":5512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2756,"num_write_kb":11024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2719","reported_seq":2773,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.401683+0000","last_change":"2024-09-16T04:49:08.115384+0000","last_active":"2024-09-16T04:49:21.401683+0000","last_peered":"2024-09-16T04:49:21.401683+0000","last_clean":"2024-09-16T04:49:21.401683+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:21.401683+0000","last_undegraded":"2024-09-16T04:49:21.401683+0000","last_fullsized":"2024-09-16T04:49:21.401683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2719,"log_dups_size":0,"ondisk_log_size":2719,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:24.326922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11113682,"num_objects":2715,"num_object_clones":0,"num_object_copies":5430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2717,"num_write_kb":10856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2684","reported_seq":2751,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.412341+0000","last_change":"2024-09-16T04:49:08.647868+0000","last_active":"2024-09-16T04:49:21.412341+0000","last_peered":"2024-09-16T04:49:21.412341+0000","last_clean":"2024-09-16T04:49:21.412341+0000","last_became_active":"2024-09-16T04:48:46.453489+0000","last_became_peered":"2024-09-16T04:48:46.453489+0000","last_unstale":"2024-09-16T04:49:21.412341+0000","last_undegraded":"2024-09-16T04:49:21.412341+0000","last_fullsized":"2024-09-16T04:49:21.412341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2684,"log_dups_size":0,"ondisk_log_size":2684,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:25.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10977897,"num_objects":2681,"num_object_clones":0,"num_object_copies":5362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2683,"num_write_kb":10722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2689","reported_seq":2736,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.452173+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:21.452173+0000","last_peered":"2024-09-16T04:49:21.452173+0000","last_clean":"2024-09-16T04:49:21.452173+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:21.452173+0000","last_undegraded":"2024-09-16T04:49:21.452173+0000","last_fullsized":"2024-09-16T04:49:21.452173+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2689,"log_dups_size":0,"ondisk_log_size":2689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11014144,"num_objects":2689,"num_object_clones":0,"num_object_copies":5378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2689,"num_write_kb":10756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2738","reported_seq":2804,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.989670+0000","last_change":"2024-09-16T04:49:21.197330+0000","last_active":"2024-09-16T04:49:21.989670+0000","last_peered":"2024-09-16T04:49:21.989670+0000","last_clean":"2024-09-16T04:49:21.989670+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:21.989670+0000","last_undegraded":"2024-09-16T04:49:21.989670+0000","last_fullsized":"2024-09-16T04:49:21.989670+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2738,"log_dups_size":0,"ondisk_log_size":2738,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11187410,"num_objects":2733,"num_object_clones":0,"num_object_copies":5466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2736,"num_write_kb":10928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2817","reported_seq":2881,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.977174+0000","last_change":"2024-09-16T04:49:21.267316+0000","last_active":"2024-09-16T04:49:21.977174+0000","last_peered":"2024-09-16T04:49:21.977174+0000","last_clean":"2024-09-16T04:49:21.977174+0000","last_became_active":"2024-09-16T04:48:45.441846+0000","last_became_peered":"2024-09-16T04:48:45.441846+0000","last_unstale":"2024-09-16T04:49:21.977174+0000","last_undegraded":"2024-09-16T04:49:21.977174+0000","last_fullsized":"2024-09-16T04:49:21.977174+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2817,"log_dups_size":0,"ondisk_log_size":2817,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11519186,"num_objects":2814,"num_object_clones":0,"num_object_copies":5628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2815,"num_write_kb":11251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2804","reported_seq":2868,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.952886+0000","last_change":"2024-09-16T04:49:21.549436+0000","last_active":"2024-09-16T04:49:21.952886+0000","last_peered":"2024-09-16T04:49:21.952886+0000","last_clean":"2024-09-16T04:49:21.952886+0000","last_became_active":"2024-09-16T04:48:45.444639+0000","last_became_peered":"2024-09-16T04:48:45.444639+0000","last_unstale":"2024-09-16T04:49:21.952886+0000","last_undegraded":"2024-09-16T04:49:21.952886+0000","last_fullsized":"2024-09-16T04:49:21.952886+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2804,"log_dups_size":0,"ondisk_log_size":2804,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11461842,"num_objects":2800,"num_object_clones":0,"num_object_copies":5600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2802,"num_write_kb":11196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2700","reported_seq":2767,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.984170+0000","last_change":"2024-09-16T04:49:21.221881+0000","last_active":"2024-09-16T04:49:21.984170+0000","last_peered":"2024-09-16T04:49:21.984170+0000","last_clean":"2024-09-16T04:49:21.984170+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:21.984170+0000","last_undegraded":"2024-09-16T04:49:21.984170+0000","last_fullsized":"2024-09-16T04:49:21.984170+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2700,"log_dups_size":0,"ondisk_log_size":2700,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11051625,"num_objects":2699,"num_object_clones":0,"num_object_copies":5398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2699,"num_write_kb":10793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449263+0000","last_change":"2024-09-16T04:48:45.440924+0000","last_active":"2024-09-16T04:48:46.449263+0000","last_peered":"2024-09-16T04:48:46.449263+0000","last_clean":"2024-09-16T04:48:46.449263+0000","last_became_active":"2024-09-16T04:48:45.440123+0000","last_became_peered":"2024-09-16T04:48:45.440123+0000","last_unstale":"2024-09-16T04:48:46.449263+0000","last_undegraded":"2024-09-16T04:48:46.449263+0000","last_fullsized":"2024-09-16T04:48:46.449263+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.338015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2660","reported_seq":2729,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.423681+0000","last_change":"2024-09-16T04:49:11.485127+0000","last_active":"2024-09-16T04:49:21.423681+0000","last_peered":"2024-09-16T04:49:21.423681+0000","last_clean":"2024-09-16T04:49:21.423681+0000","last_became_active":"2024-09-16T04:48:46.454004+0000","last_became_peered":"2024-09-16T04:48:46.454004+0000","last_unstale":"2024-09-16T04:49:21.423681+0000","last_undegraded":"2024-09-16T04:49:21.423681+0000","last_fullsized":"2024-09-16T04:49:21.423681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2660,"log_dups_size":0,"ondisk_log_size":2660,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:22.363628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10867922,"num_objects":2655,"num_object_clones":0,"num_object_copies":5310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2658,"num_write_kb":10616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.454453+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:48:46.454453+0000","last_peered":"2024-09-16T04:48:46.454453+0000","last_clean":"2024-09-16T04:48:46.454453+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:48:46.454453+0000","last_undegraded":"2024-09-16T04:48:46.454453+0000","last_fullsized":"2024-09-16T04:48:46.454453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2852","reported_seq":2911,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.991411+0000","last_change":"2024-09-16T04:49:21.514543+0000","last_active":"2024-09-16T04:49:21.991411+0000","last_peered":"2024-09-16T04:49:21.991411+0000","last_clean":"2024-09-16T04:49:21.991411+0000","last_became_active":"2024-09-16T04:48:45.444610+0000","last_became_peered":"2024-09-16T04:48:45.444610+0000","last_unstale":"2024-09-16T04:49:21.991411+0000","last_undegraded":"2024-09-16T04:49:21.991411+0000","last_fullsized":"2024-09-16T04:49:21.991411+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2852,"log_dups_size":0,"ondisk_log_size":2852,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11650258,"num_objects":2846,"num_object_clones":0,"num_object_copies":5692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2850,"num_write_kb":11380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":67,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.735224+0000","last_change":"2024-09-16T04:49:21.735224+0000","last_active":"2024-09-16T04:49:21.735224+0000","last_peered":"2024-09-16T04:49:21.735224+0000","last_clean":"2024-09-16T04:49:21.735224+0000","last_became_active":"2024-09-16T04:48:45.440291+0000","last_became_peered":"2024-09-16T04:48:45.440291+0000","last_unstale":"2024-09-16T04:49:21.735224+0000","last_undegraded":"2024-09-16T04:49:21.735224+0000","last_fullsized":"2024-09-16T04:49:21.735224+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:06.118479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2651","reported_seq":2710,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.441117+0000","last_change":"2024-09-16T04:49:12.621094+0000","last_active":"2024-09-16T04:49:21.441117+0000","last_peered":"2024-09-16T04:49:21.441117+0000","last_clean":"2024-09-16T04:49:21.441117+0000","last_became_active":"2024-09-16T04:48:46.452651+0000","last_became_peered":"2024-09-16T04:48:46.452651+0000","last_unstale":"2024-09-16T04:49:21.441117+0000","last_undegraded":"2024-09-16T04:49:21.441117+0000","last_fullsized":"2024-09-16T04:49:21.441117+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2651,"log_dups_size":0,"ondisk_log_size":2651,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:23.362822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10839250,"num_objects":2648,"num_object_clones":0,"num_object_copies":5296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2649,"num_write_kb":10587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.454753+0000","last_change":"2024-09-16T04:48:46.454753+0000","last_active":"2024-09-16T04:48:46.454753+0000","last_peered":"2024-09-16T04:48:46.454753+0000","last_clean":"2024-09-16T04:48:46.454753+0000","last_became_active":"2024-09-16T04:48:46.454400+0000","last_became_peered":"2024-09-16T04:48:46.454400+0000","last_unstale":"2024-09-16T04:48:46.454753+0000","last_undegraded":"2024-09-16T04:48:46.454753+0000","last_fullsized":"2024-09-16T04:48:46.454753+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:50:39.528170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2699","reported_seq":2758,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.430184+0000","last_change":"2024-09-16T04:49:13.167289+0000","last_active":"2024-09-16T04:49:21.430184+0000","last_peered":"2024-09-16T04:49:21.430184+0000","last_clean":"2024-09-16T04:49:21.430184+0000","last_became_active":"2024-09-16T04:48:46.454141+0000","last_became_peered":"2024-09-16T04:48:46.454141+0000","last_unstale":"2024-09-16T04:49:21.430184+0000","last_undegraded":"2024-09-16T04:49:21.430184+0000","last_fullsized":"2024-09-16T04:49:21.430184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2699,"log_dups_size":0,"ondisk_log_size":2699,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:23.362847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11031762,"num_objects":2695,"num_object_clones":0,"num_object_copies":5390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2697,"num_write_kb":10775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2756","reported_seq":2817,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.993279+0000","last_change":"2024-09-16T04:49:21.592366+0000","last_active":"2024-09-16T04:49:21.993279+0000","last_peered":"2024-09-16T04:49:21.993279+0000","last_clean":"2024-09-16T04:49:21.993279+0000","last_became_active":"2024-09-16T04:48:45.444748+0000","last_became_peered":"2024-09-16T04:48:45.444748+0000","last_unstale":"2024-09-16T04:49:21.993279+0000","last_undegraded":"2024-09-16T04:49:21.993279+0000","last_fullsized":"2024-09-16T04:49:21.993279+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2756,"log_dups_size":0,"ondisk_log_size":2756,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11281001,"num_objects":2755,"num_object_clones":0,"num_object_copies":5510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2755,"num_write_kb":11017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449472+0000","last_change":"2024-09-16T04:48:45.443815+0000","last_active":"2024-09-16T04:48:46.449472+0000","last_peered":"2024-09-16T04:48:46.449472+0000","last_clean":"2024-09-16T04:48:46.449472+0000","last_became_active":"2024-09-16T04:48:45.443064+0000","last_became_peered":"2024-09-16T04:48:45.443064+0000","last_unstale":"2024-09-16T04:48:46.449472+0000","last_undegraded":"2024-09-16T04:48:46.449472+0000","last_fullsized":"2024-09-16T04:48:46.449472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:25:35.280081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2876","reported_seq":2942,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.989943+0000","last_change":"2024-09-16T04:49:21.706144+0000","last_active":"2024-09-16T04:49:21.989943+0000","last_peered":"2024-09-16T04:49:21.989943+0000","last_clean":"2024-09-16T04:49:21.989943+0000","last_became_active":"2024-09-16T04:48:45.445679+0000","last_became_peered":"2024-09-16T04:48:45.445679+0000","last_unstale":"2024-09-16T04:49:21.989943+0000","last_undegraded":"2024-09-16T04:49:21.989943+0000","last_fullsized":"2024-09-16T04:49:21.989943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2876,"log_dups_size":0,"ondisk_log_size":2876,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11749179,"num_objects":2871,"num_object_clones":0,"num_object_copies":5742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2873,"num_write_kb":11476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.453526+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:48:46.453526+0000","last_peered":"2024-09-16T04:48:46.453526+0000","last_clean":"2024-09-16T04:48:46.453526+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:48:46.453526+0000","last_undegraded":"2024-09-16T04:48:46.453526+0000","last_fullsized":"2024-09-16T04:48:46.453526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2689","reported_seq":2752,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.979862+0000","last_change":"2024-09-16T04:49:21.354274+0000","last_active":"2024-09-16T04:49:21.979862+0000","last_peered":"2024-09-16T04:49:21.979862+0000","last_clean":"2024-09-16T04:49:21.979862+0000","last_became_active":"2024-09-16T04:48:45.444067+0000","last_became_peered":"2024-09-16T04:48:45.444067+0000","last_unstale":"2024-09-16T04:49:21.979862+0000","last_undegraded":"2024-09-16T04:49:21.979862+0000","last_fullsized":"2024-09-16T04:49:21.979862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2689,"log_dups_size":0,"ondisk_log_size":2689,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10998994,"num_objects":2687,"num_object_clones":0,"num_object_copies":5374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.449269+0000","last_change":"2024-09-16T04:48:45.437264+0000","last_active":"2024-09-16T04:48:46.449269+0000","last_peered":"2024-09-16T04:48:46.449269+0000","last_clean":"2024-09-16T04:48:46.449269+0000","last_became_active":"2024-09-16T04:48:45.435646+0000","last_became_peered":"2024-09-16T04:48:45.435646+0000","last_unstale":"2024-09-16T04:48:46.449269+0000","last_undegraded":"2024-09-16T04:48:46.449269+0000","last_fullsized":"2024-09-16T04:48:46.449269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:03:29.272297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:48:46.453655+0000","last_change":"2024-09-16T04:48:46.453655+0000","last_active":"2024-09-16T04:48:46.453655+0000","last_peered":"2024-09-16T04:48:46.453655+0000","last_clean":"2024-09-16T04:48:46.453655+0000","last_became_active":"2024-09-16T04:48:46.453374+0000","last_became_peered":"2024-09-16T04:48:46.453374+0000","last_unstale":"2024-09-16T04:48:46.453655+0000","last_undegraded":"2024-09-16T04:48:46.453655+0000","last_fullsized":"2024-09-16T04:48:46.453655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:49:42.225940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2916","reported_seq":2985,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T04:49:21.954188+0000","last_change":"2024-09-16T04:49:21.456239+0000","last_active":"2024-09-16T04:49:21.954188+0000","last_peered":"2024-09-16T04:49:21.954188+0000","last_clean":"2024-09-16T04:49:21.954188+0000","last_became_active":"2024-09-16T04:48:45.440990+0000","last_became_peered":"2024-09-16T04:48:45.440990+0000","last_unstale":"2024-09-16T04:49:21.954188+0000","last_undegraded":"2024-09-16T04:49:21.954188+0000","last_fullsized":"2024-09-16T04:49:21.954188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":2916,"log_dups_size":0,"ondisk_log_size":2916,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11897252,"num_objects":2908,"num_object_clones":0,"num_object_copies":5816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2913,"num_write_kb":11623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":617,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":179929980,"num_objects":43952,"num_object_clones":0,"num_object_copies":87904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43979,"num_write_kb":175742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":617,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":359301120,"data_stored":359196750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44006,"ondisk_log_size":44006,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986188,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":281388,"kb_used_data":173340,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94090452,"statfs":{"total":96636764160,"available":96348622848,"internally_reserved":0,"allocated":177500160,"data_stored":177291584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":110620329},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182417,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":284416,"kb_used_data":172272,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94087424,"statfs":{"total":96636764160,"available":96345522176,"internally_reserved":0,"allocated":176406528,"data_stored":176173001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114814634},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,4,6,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":181174272,"data_stored":181170793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":178012160,"data_stored":178008681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:24.467 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-16T04:49:24.467 INFO:tasks.util.rados:rados -p unfounddel_pool get new_1 - 2024-09-16T04:49:24.467 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_1 - 2024-09-16T04:49:24.575 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/new_1: (2) No such file or directory 2024-09-16T04:49:24.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:24.587 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_1 - 2024-09-16T04:49:24.587 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_1 - 2024-09-16T04:49:24.659 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existed_1: (2) No such file or directory 2024-09-16T04:49:24.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:24.672 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_1 - 2024-09-16T04:49:24.672 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_1 - 2024-09-16T04:49:24.779 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existing_1: (2) No such file or directory 2024-09-16T04:49:24.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:24.784 INFO:tasks.util.rados:rados -p unfounddel_pool get new_2 - 2024-09-16T04:49:24.784 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_2 - 2024-09-16T04:49:24.860 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/new_2: (2) No such file or directory 2024-09-16T04:49:24.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:24.876 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_2 - 2024-09-16T04:49:24.876 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_2 - 2024-09-16T04:49:24.996 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existed_2: (2) No such file or directory 2024-09-16T04:49:25.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:25.013 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_2 - 2024-09-16T04:49:25.013 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_2 - 2024-09-16T04:49:25.050 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 28 20 47678 47658 6.64753 3.60547 0.0452412 0.0117401 2024-09-16T04:49:25.102 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existing_2: (2) No such file or directory 2024-09-16T04:49:25.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:25.111 INFO:tasks.util.rados:rados -p unfounddel_pool get new_3 - 2024-09-16T04:49:25.111 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_3 - 2024-09-16T04:49:25.244 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/new_3: (2) No such file or directory 2024-09-16T04:49:25.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:25.267 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_3 - 2024-09-16T04:49:25.267 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_3 - 2024-09-16T04:49:25.355 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existed_3: (2) No such file or directory 2024-09-16T04:49:25.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:25.360 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_3 - 2024-09-16T04:49:25.360 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_3 - 2024-09-16T04:49:25.448 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existing_3: (2) No such file or directory 2024-09-16T04:49:25.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:25.458 INFO:tasks.util.rados:rados -p unfounddel_pool get new_4 - 2024-09-16T04:49:25.459 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_4 - 2024-09-16T04:49:25.568 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/new_4: (2) No such file or directory 2024-09-16T04:49:25.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:25.572 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_4 - 2024-09-16T04:49:25.572 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_4 - 2024-09-16T04:49:25.653 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existed_4: (2) No such file or directory 2024-09-16T04:49:25.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:25.659 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_4 - 2024-09-16T04:49:25.659 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_4 - 2024-09-16T04:49:25.720 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existing_4: (2) No such file or directory 2024-09-16T04:49:25.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:25.738 INFO:tasks.util.rados:rados -p unfounddel_pool get new_5 - 2024-09-16T04:49:25.738 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_5 - 2024-09-16T04:49:25.814 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/new_5: (2) No such file or directory 2024-09-16T04:49:25.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:25.821 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_5 - 2024-09-16T04:49:25.822 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_5 - 2024-09-16T04:49:25.921 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existed_5: (2) No such file or directory 2024-09-16T04:49:25.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:25.924 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_5 - 2024-09-16T04:49:25.924 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_5 - 2024-09-16T04:49:26.010 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existing_5: (2) No such file or directory 2024-09-16T04:49:26.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:26.019 INFO:tasks.util.rados:rados -p unfounddel_pool get new_6 - 2024-09-16T04:49:26.019 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_6 - 2024-09-16T04:49:26.051 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 29 20 48510 48490 6.53038 3.25 0.023957 0.0119505 2024-09-16T04:49:26.101 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/new_6: (2) No such file or directory 2024-09-16T04:49:26.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:26.108 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_6 - 2024-09-16T04:49:26.108 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_6 - 2024-09-16T04:49:26.238 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existed_6: (2) No such file or directory 2024-09-16T04:49:26.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:26.245 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_6 - 2024-09-16T04:49:26.246 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_6 - 2024-09-16T04:49:26.329 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existing_6: (2) No such file or directory 2024-09-16T04:49:26.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:26.336 INFO:tasks.util.rados:rados -p unfounddel_pool get new_7 - 2024-09-16T04:49:26.337 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_7 - 2024-09-16T04:49:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/new_7: (2) No such file or directory 2024-09-16T04:49:26.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:26.463 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_7 - 2024-09-16T04:49:26.463 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_7 - 2024-09-16T04:49:26.531 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existed_7: (2) No such file or directory 2024-09-16T04:49:26.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:26.547 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_7 - 2024-09-16T04:49:26.547 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_7 - 2024-09-16T04:49:26.638 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existing_7: (2) No such file or directory 2024-09-16T04:49:26.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:26.645 INFO:tasks.util.rados:rados -p unfounddel_pool get new_8 - 2024-09-16T04:49:26.645 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_8 - 2024-09-16T04:49:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/new_8: (2) No such file or directory 2024-09-16T04:49:26.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:26.747 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_8 - 2024-09-16T04:49:26.747 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_8 - 2024-09-16T04:49:26.820 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existed_8: (2) No such file or directory 2024-09-16T04:49:26.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:26.828 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_8 - 2024-09-16T04:49:26.828 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_8 - 2024-09-16T04:49:26.911 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existing_8: (2) No such file or directory 2024-09-16T04:49:26.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:26.926 INFO:tasks.util.rados:rados -p unfounddel_pool get new_9 - 2024-09-16T04:49:26.926 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_9 - 2024-09-16T04:49:27.028 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/new_9: (2) No such file or directory 2024-09-16T04:49:27.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:27.044 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_9 - 2024-09-16T04:49:27.044 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_9 - 2024-09-16T04:49:27.050 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 30 20 49376 49356 6.42546 3.38281 0.012513 0.0121514 2024-09-16T04:49:27.136 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existed_9: (2) No such file or directory 2024-09-16T04:49:27.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:27.141 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_9 - 2024-09-16T04:49:27.141 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_9 - 2024-09-16T04:49:27.244 INFO:teuthology.orchestra.run.smithi012.stderr:error getting unfounddel_pool/existing_9: (2) No such file or directory 2024-09-16T04:49:27.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T04:49:27.252 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-16T04:49:27.904 INFO:tasks.ceph.mon.c.smithi012.stderr:2024-09-16T04:49:27.896+0000 7fbe3b864640 -1 mon.c@2(leader).osd e36 definitely_dead 0 2024-09-16T04:49:28.051 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 31 20 50221 50201 6.32466 3.30078 0.0124377 0.012345 2024-09-16T04:49:28.574 INFO:teuthology.orchestra.run.smithi012.stderr:marked in osd.1. 2024-09-16T04:49:28.613 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T04:49:28.613 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T04:49:28.615 INFO:tasks.ceph.osd.1:Started 2024-09-16T04:49:28.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T04:49:28.721 INFO:teuthology.orchestra.run.smithi012.stderr:no valid command found; 10 closest matches: 2024-09-16T04:49:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:0 2024-09-16T04:49:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:1 2024-09-16T04:49:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:2 2024-09-16T04:49:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:abort 2024-09-16T04:49:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:assert 2024-09-16T04:49:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:config diff 2024-09-16T04:49:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:config diff get 2024-09-16T04:49:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:config get 2024-09-16T04:49:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:config help [] 2024-09-16T04:49:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:config set ... 2024-09-16T04:49:28.722 INFO:teuthology.orchestra.run.smithi012.stderr:admin_socket: invalid command 2024-09-16T04:49:28.724 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T04:49:28.724 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-16T04:49:29.051 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 32 20 50748 50728 6.19135 2.05859 0.0059633 0.0124325 2024-09-16T04:49:30.051 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 33 20 50748 50728 6.00373 0 - 0.0124325 2024-09-16T04:49:30.415 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-16T04:49:30.408+0000 7fb346d937c0 -1 Falling back to public interface 2024-09-16T04:49:31.051 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 34 20 51223 51203 5.88173 0.927734 0.00704751 0.0132763 2024-09-16T04:49:32.051 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 35 20 52342 52322 5.83856 4.37109 0.0237751 0.0133744 2024-09-16T04:49:33.051 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 36 20 53080 53060 5.75645 2.88281 0.0032203 0.0134387 2024-09-16T04:49:33.725 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T04:49:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:no valid command found; 10 closest matches: 2024-09-16T04:49:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:0 2024-09-16T04:49:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:1 2024-09-16T04:49:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:2 2024-09-16T04:49:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:abort 2024-09-16T04:49:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:assert 2024-09-16T04:49:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:config diff 2024-09-16T04:49:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:config diff get 2024-09-16T04:49:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:config get 2024-09-16T04:49:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:config help [] 2024-09-16T04:49:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:config set ... 2024-09-16T04:49:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:admin_socket: invalid command 2024-09-16T04:49:33.845 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T04:49:33.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-16T04:49:34.051 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 37 20 53080 53060 5.60088 0 - 0.0134387 2024-09-16T04:49:34.223 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-16T04:49:34.216+0000 7fb346d937c0 -1 osd.1 26 log_to_monitors true 2024-09-16T04:49:34.638 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-16T04:49:34.632+0000 7fb33ab60640 -1 osd.1 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T04:49:35.051 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 38 20 53508 53488 5.49749 0.835938 0.0110053 0.0142027 2024-09-16T04:49:36.052 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 39 20 54142 54122 5.42002 2.47656 0.0025026 0.0142594 2024-09-16T04:49:37.052 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:2024-09-16T04:49:37.051271+0000 min lat: 0.00206892 max lat: 2.11704 avg lat: 0.0142594 2024-09-16T04:49:37.052 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T04:49:37.052 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 40 20 54142 54122 5.28453 0 - 0.0142594 2024-09-16T04:49:38.052 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 41 20 54143 54123 5.15573 0.00195312 2.40288 0.0143035 2024-09-16T04:49:38.847 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T04:49:39.053 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 42 20 54707 54687 5.08543 2.20312 0.0614692 0.0153455 2024-09-16T04:49:39.088 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T04:49:39.089 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-16T04:49:39.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.089 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.35177.0:54692 2.2 e45/44 2:4bc6da1d:::benchmark_data_smithi012_34635_object54691:head v 45'3517, mlcod=23'6)", 2024-09-16T04:49:39.089 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-16T04:49:39.047682+0000", 2024-09-16T04:49:39.089 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.039128209999999997, 2024-09-16T04:49:39.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.039151868999999999, 2024-09-16T04:49:39.089 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-09-16T04:49:39.089 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-16T04:49:39.089 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-16T04:49:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-16T04:49:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-16T04:49:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.047682+0000", 2024-09-16T04:49:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-16T04:49:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.047682+0000", 2024-09-16T04:49:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-16T04:49:39.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.047683+0000", 2024-09-16T04:49:39.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3400000000000001e-06 2024-09-16T04:49:39.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-16T04:49:39.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.047693+0000", 2024-09-16T04:49:39.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0108999999999999e-05 2024-09-16T04:49:39.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.092 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-16T04:49:39.092 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.047703+0000", 2024-09-16T04:49:39.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0118999999999999e-05 2024-09-16T04:49:39.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.092 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-16T04:49:39.092 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.047711+0000", 2024-09-16T04:49:39.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5490000000000001e-06 2024-09-16T04:49:39.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.092 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-16T04:49:39.092 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.066230+0000", 2024-09-16T04:49:39.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.018519537999999999 2024-09-16T04:49:39.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-16T04:49:39.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.066272+0000", 2024-09-16T04:49:39.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1483000000000002e-05 2024-09-16T04:49:39.093 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.093 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-16T04:49:39.093 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.093 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.35177.0:54696 2.2 e45/44 2:49bee7fb:::benchmark_data_smithi012_34635_object54695:head v 45'3518, mlcod=23'6)", 2024-09-16T04:49:39.093 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-16T04:49:39.050431+0000", 2024-09-16T04:49:39.094 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.036379278000000001, 2024-09-16T04:49:39.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.036468225999999999, 2024-09-16T04:49:39.094 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-09-16T04:49:39.094 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-16T04:49:39.094 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-16T04:49:39.094 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-16T04:49:39.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-16T04:49:39.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.050431+0000", 2024-09-16T04:49:39.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-16T04:49:39.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.050431+0000", 2024-09-16T04:49:39.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-16T04:49:39.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.050432+0000", 2024-09-16T04:49:39.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4410000000000001e-06 2024-09-16T04:49:39.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.096 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-16T04:49:39.096 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.050442+0000", 2024-09-16T04:49:39.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0247e-05 2024-09-16T04:49:39.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.096 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-16T04:49:39.096 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.050451+0000", 2024-09-16T04:49:39.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7139999999999999e-06 2024-09-16T04:49:39.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.096 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-16T04:49:39.096 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.050458+0000", 2024-09-16T04:49:39.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4699999999999999e-06 2024-09-16T04:49:39.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.097 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-16T04:49:39.097 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.070898+0000", 2024-09-16T04:49:39.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.020440341000000001 2024-09-16T04:49:39.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.097 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-16T04:49:39.097 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.070963+0000", 2024-09-16T04:49:39.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5388000000000005e-05 2024-09-16T04:49:39.099 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.099 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-16T04:49:39.099 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.100 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.35177.0:54713 2.a e45/44 2:5242a55d:::benchmark_data_smithi012_34635_object54712:head v 45'3496, mlcod=23'3)", 2024-09-16T04:49:39.100 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-16T04:49:39.080441+0000", 2024-09-16T04:49:39.100 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0063686819999999996, 2024-09-16T04:49:39.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0065145330000000003, 2024-09-16T04:49:39.101 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-09-16T04:49:39.101 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-16T04:49:39.101 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-16T04:49:39.101 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-16T04:49:39.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-16T04:49:39.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.080441+0000", 2024-09-16T04:49:39.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-16T04:49:39.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.080441+0000", 2024-09-16T04:49:39.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-16T04:49:39.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.080444+0000", 2024-09-16T04:49:39.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2510000000000001e-06 2024-09-16T04:49:39.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-16T04:49:39.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.080457+0000", 2024-09-16T04:49:39.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3001e-05 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.080472+0000", 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5288e-05 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.080484+0000", 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2308e-05 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-16T04:49:39.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086210+0000", 2024-09-16T04:49:39.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0057254699999999999 2024-09-16T04:49:39.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-16T04:49:39.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086239+0000", 2024-09-16T04:49:39.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9295999999999999e-05 2024-09-16T04:49:39.104 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.104 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-16T04:49:39.104 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.35177.0:54719 2.6 e45/44 2:65c397e9:::benchmark_data_smithi012_34635_object54718:head v 45'3563, mlcod=23'5)", 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-16T04:49:39.080611+0000", 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0061987869999999999, 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0063988830000000002, 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.080611+0000", 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-16T04:49:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.080611+0000", 2024-09-16T04:49:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-16T04:49:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.080613+0000", 2024-09-16T04:49:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3829999999999999e-06 2024-09-16T04:49:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-16T04:49:39.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.080625+0000", 2024-09-16T04:49:39.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2522e-05 2024-09-16T04:49:39.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-16T04:49:39.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.080638+0000", 2024-09-16T04:49:39.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2789000000000001e-05 2024-09-16T04:49:39.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-16T04:49:39.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.080644+0000", 2024-09-16T04:49:39.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.387e-06 2024-09-16T04:49:39.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.108 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-16T04:49:39.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.081991+0000", 2024-09-16T04:49:39.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001346763 2024-09-16T04:49:39.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.108 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-16T04:49:39.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.082453+0000", 2024-09-16T04:49:39.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046184200000000001 2024-09-16T04:49:39.108 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.108 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-16T04:49:39.108 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.35177.0:54727 2.7 e45/44 2:ed85cd38:::benchmark_data_smithi012_34635_object54726:head v 45'3451, mlcod=23'1)", 2024-09-16T04:49:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-16T04:49:39.083357+0000", 2024-09-16T04:49:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0034527109999999998, 2024-09-16T04:49:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003709637, 2024-09-16T04:49:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-09-16T04:49:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-16T04:49:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-16T04:49:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-16T04:49:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-16T04:49:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083357+0000", 2024-09-16T04:49:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083357+0000", 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083359+0000", 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.44e-06 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083374+0000", 2024-09-16T04:49:39.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5523e-05 2024-09-16T04:49:39.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-16T04:49:39.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083384+0000", 2024-09-16T04:49:39.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6420000000000002e-06 2024-09-16T04:49:39.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-16T04:49:39.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083392+0000", 2024-09-16T04:49:39.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6890000000000004e-06 2024-09-16T04:49:39.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-16T04:49:39.112 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083481+0000", 2024-09-16T04:49:39.112 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9239000000000004e-05 2024-09-16T04:49:39.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.112 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-16T04:49:39.112 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083505+0000", 2024-09-16T04:49:39.112 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3808999999999999e-05 2024-09-16T04:49:39.112 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.112 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-16T04:49:39.112 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.112 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.35177.0:54724 2.2 e45/44)", 2024-09-16T04:49:39.113 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-16T04:49:39.083594+0000", 2024-09-16T04:49:39.113 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0032157729999999999, 2024-09-16T04:49:39.113 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0035249529999999999, 2024-09-16T04:49:39.113 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-09-16T04:49:39.113 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-16T04:49:39.113 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-16T04:49:39.113 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-16T04:49:39.113 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.113 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-16T04:49:39.113 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083594+0000", 2024-09-16T04:49:39.113 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.113 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-16T04:49:39.114 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083594+0000", 2024-09-16T04:49:39.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-16T04:49:39.114 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083596+0000", 2024-09-16T04:49:39.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4419999999999999e-06 2024-09-16T04:49:39.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-16T04:49:39.114 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083606+0000", 2024-09-16T04:49:39.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0083e-05 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083615+0000", 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1509999999999993e-06 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.083624+0000", 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0669999999999998e-06 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDPGPush(2.0 45/44 [PushOp(2:0e1c2785:::benchmark_data_smithi012_34635_object488:head, version: 36'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:0e1c2785:::benchmark_data_smithi012_34635_object488:head@36'30, size: 4096, copy_subset: [(0, 4096)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-16T04:49:39.086710+0000", 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.9568999999999995e-05, 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00047572499999999998, 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086710+0000", 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-16T04:49:39.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086710+0000", 2024-09-16T04:49:39.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-16T04:49:39.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086713+0000", 2024-09-16T04:49:39.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-16T04:49:39.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-16T04:49:39.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086732+0000", 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9327e-05 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086759+0000", 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7124999999999999e-05 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086779+0000", 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9687999999999999e-05 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-16T04:49:39.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086874+0000", 2024-09-16T04:49:39.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4809999999999995e-05 2024-09-16T04:49:39.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.119 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-16T04:49:39.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086889+0000", 2024-09-16T04:49:39.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4586999999999999e-05 2024-09-16T04:49:39.119 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.119 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-16T04:49:39.119 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.119 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.35177.0:54725 2.2 e45/44)", 2024-09-16T04:49:39.120 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-16T04:49:39.086879+0000", 2024-09-16T04:49:39.120 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4294967295.9999313, 2024-09-16T04:49:39.120 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000359389, 2024-09-16T04:49:39.120 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-09-16T04:49:39.120 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-16T04:49:39.120 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-16T04:49:39.120 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-16T04:49:39.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.120 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-16T04:49:39.120 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086879+0000", 2024-09-16T04:49:39.120 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-16T04:49:39.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086879+0000", 2024-09-16T04:49:39.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-16T04:49:39.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086880+0000", 2024-09-16T04:49:39.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.384e-06 2024-09-16T04:49:39.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.122 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-16T04:49:39.122 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086894+0000", 2024-09-16T04:49:39.122 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4134000000000001e-05 2024-09-16T04:49:39.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.122 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-16T04:49:39.122 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086904+0000", 2024-09-16T04:49:39.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8120000000000002e-06 2024-09-16T04:49:39.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.123 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-16T04:49:39.123 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086911+0000", 2024-09-16T04:49:39.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.562e-06 2024-09-16T04:49:39.123 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.123 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-16T04:49:39.123 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.123 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.35177.0:54689 2.2 e45/44 2:4ac9e732:::benchmark_data_smithi012_34635_object54688:head v 45'3516, mlcod=23'6)", 2024-09-16T04:49:39.123 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-16T04:49:39.043706+0000", 2024-09-16T04:49:39.124 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.043104112999999999, 2024-09-16T04:49:39.124 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.043579037000000001, 2024-09-16T04:49:39.124 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-09-16T04:49:39.124 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-16T04:49:39.124 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T04:49:39.124 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-16T04:49:39.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.124 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-16T04:49:39.124 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.043706+0000", 2024-09-16T04:49:39.124 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.124 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-16T04:49:39.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.043706+0000", 2024-09-16T04:49:39.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-16T04:49:39.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.125 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-16T04:49:39.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.043707+0000", 2024-09-16T04:49:39.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5260000000000001e-06 2024-09-16T04:49:39.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.125 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-16T04:49:39.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.043721+0000", 2024-09-16T04:49:39.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3499000000000001e-05 2024-09-16T04:49:39.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.043734+0000", 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2889e-05 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.043744+0000", 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8600000000000005e-06 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.052600+0000", 2024-09-16T04:49:39.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0088560189999999997 2024-09-16T04:49:39.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-16T04:49:39.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.052626+0000", 2024-09-16T04:49:39.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6069999999999999e-05 2024-09-16T04:49:39.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-16T04:49:39.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-16T04:49:39.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-16T04:49:39.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-16T04:49:39.086955+0000", 2024-09-16T04:49:39.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.034329408999999998 2024-09-16T04:49:39.127 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.127 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-16T04:49:39.128 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.128 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-16T04:49:39.128 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-16T04:49:39.128 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 9 2024-09-16T04:49:39.128 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T04:49:39.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.1 to be up 2024-09-16T04:49:39.129 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T04:49:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:39.987 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":45,"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","created":"2024-09-16T04:47:27.160076+0000","modified":"2024-09-16T04:49:37.632987+0000","last_up_change":"2024-09-16T04:49:35.617843+0000","last_in_change":"2024-09-16T04:49:28.546152+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-16T04:47:42.612259+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-16T04:47:55.339654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7fa4b8a1-d218-47cb-adc4-469674c829fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":21,"up_from":25,"up_thru":44,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6800","nonce":2560862748}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6801","nonce":2560862748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6803","nonce":2560862748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6802","nonce":2560862748}]},"public_addr":"172.21.15.12:6800/2560862748","cluster_addr":"172.21.15.12:6801/2560862748","heartbeat_back_addr":"172.21.15.12:6803/2560862748","heartbeat_front_addr":"172.21.15.12:6802/2560862748","state":["exists","up"]},{"osd":1,"uuid":"7234f3ae-2cf3-4dd4-8c21-4f170976dbf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":43,"up_thru":44,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6808","nonce":183716785}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6809","nonce":183716785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6811","nonce":183716785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6810","nonce":183716785}]},"public_addr":"172.21.15.12:6808/183716785","cluster_addr":"172.21.15.12:6809/183716785","heartbeat_back_addr":"172.21.15.12:6811/183716785","heartbeat_front_addr":"172.21.15.12:6810/183716785","state":["exists","up"]},{"osd":2,"uuid":"919a351e-ecee-4e06-bff1-ae68c80d320f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":18,"up_from":31,"up_thru":44,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6804","nonce":1526510468}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6805","nonce":1526510468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6807","nonce":1526510468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6806","nonce":1526510468}]},"public_addr":"172.21.15.12:6804/1526510468","cluster_addr":"172.21.15.12:6805/1526510468","heartbeat_back_addr":"172.21.15.12:6807/1526510468","heartbeat_front_addr":"172.21.15.12:6806/1526510468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.109010+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:35.949029+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.023975+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"1.0","mappings":[{"from":0,"to":2}]},{"pgid":"2.6","mappings":[{"from":0,"to":2}]},{"pgid":"2.7","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"2.0","osds":[2]},{"pgid":"2.1","osds":[2]},{"pgid":"2.2","osds":[0]},{"pgid":"2.3","osds":[2]},{"pgid":"2.4","osds":[0]},{"pgid":"2.5","osds":[0]},{"pgid":"2.6","osds":[2]},{"pgid":"2.7","osds":[2]},{"pgid":"2.9","osds":[0]},{"pgid":"2.a","osds":[0]},{"pgid":"2.d","osds":[2]},{"pgid":"2.f","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T04:49:40.048 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.1 is up 2024-09-16T04:49:40.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-16T04:49:40.049 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:40.052 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 43 20 55366 55346 5.02702 2.57422 0.0426032 0.0155215 2024-09-16T04:49:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:40.824 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:40.896 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-16T04:49:40.278223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":221720818,"num_objects":54139,"num_object_clones":0,"num_object_copies":108278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19373,"num_write_kb":77458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54296,"ondisk_log_size":54296,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":738404,"kb_used_data":436512,"kb_used_omap":12,"kb_used_meta":301811,"kb_avail":282377116,"statfs":{"total":289910292480,"available":289154166784,"internally_reserved":0,"allocated":446988288,"data_stored":446154948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":309054465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,6,14,6,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8144253,"num_objects":1979,"num_object_clones":0,"num_object_copies":3958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-39245,"num_objects_unfound":0,"num_objects_dirty":1979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-17696,"num_write_kb":-70707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-12,"num_bytes_recovered":-4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.653919"},"pg_stats":[{"pgid":"2.f","version":"42'3341","reported_seq":3409,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:35.654718+0000","last_change":"2024-09-16T04:49:35.654718+0000","last_active":"2024-09-16T04:49:35.654718+0000","last_peered":"2024-09-16T04:49:35.616010+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:30.661759+0000","last_became_peered":"2024-09-16T04:49:30.661759+0000","last_unstale":"2024-09-16T04:49:35.654718+0000","last_undegraded":"2024-09-16T04:49:35.654718+0000","last_fullsized":"2024-09-16T04:49:35.654718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3341,"log_dups_size":0,"ondisk_log_size":3341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13684736,"num_objects":3341,"num_object_clones":0,"num_object_copies":6682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3341,"num_write_kb":13364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"42'3362","reported_seq":3469,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-16T04:49:35.650619+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:49:35.650619+0000","last_peered":"2024-09-16T04:49:35.650619+0000","last_clean":"2024-09-16T04:49:35.650619+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:35.650619+0000","last_undegraded":"2024-09-16T04:49:35.650619+0000","last_fullsized":"2024-09-16T04:49:35.650619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":3362,"log_dups_size":0,"ondisk_log_size":3362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13746176,"num_objects":3356,"num_object_clones":0,"num_object_copies":6712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":331,"num_write_kb":1324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"42'3323","reported_seq":3448,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:35.615806+0000","last_change":"2024-09-16T04:49:35.644752+0000","last_active":"2024-09-16T04:49:35.644752+0000","last_peered":"2024-09-16T04:49:35.615806+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:30.606516+0000","last_became_peered":"2024-09-16T04:49:30.606516+0000","last_unstale":"2024-09-16T04:49:35.615806+0000","last_undegraded":"2024-09-16T04:49:35.615806+0000","last_fullsized":"2024-09-16T04:49:35.615806+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":3323,"log_dups_size":0,"ondisk_log_size":3323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13594624,"num_objects":3319,"num_object_clones":0,"num_object_copies":6638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'3397","reported_seq":3458,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-16T04:49:35.662330+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:35.662330+0000","last_peered":"2024-09-16T04:49:35.662330+0000","last_clean":"2024-09-16T04:49:35.662330+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:35.662330+0000","last_undegraded":"2024-09-16T04:49:35.662330+0000","last_fullsized":"2024-09-16T04:49:35.662330+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3397,"log_dups_size":0,"ondisk_log_size":3397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13914112,"num_objects":3397,"num_object_clones":0,"num_object_copies":6794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3397,"num_write_kb":13588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"42'3387","reported_seq":3599,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T04:49:36.640767+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:49:36.640767+0000","last_peered":"2024-09-16T04:49:36.640767+0000","last_clean":"2024-09-16T04:49:36.640767+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:36.640767+0000","last_undegraded":"2024-09-16T04:49:36.640767+0000","last_fullsized":"2024-09-16T04:49:36.640767+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":3387,"log_dups_size":0,"ondisk_log_size":3387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13750272,"num_objects":3357,"num_object_clones":0,"num_object_copies":6714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":419,"num_write_kb":1676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"42'3461","reported_seq":3583,"reported_epoch":44,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:36.641994+0000","last_change":"2024-09-16T04:49:35.657497+0000","last_active":"2024-09-16T04:49:35.657497+0000","last_peered":"2024-09-16T04:49:35.646925+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:30.638270+0000","last_became_peered":"2024-09-16T04:49:30.638270+0000","last_unstale":"2024-09-16T04:49:36.641994+0000","last_undegraded":"2024-09-16T04:49:36.641994+0000","last_fullsized":"2024-09-16T04:49:36.641994+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":3461,"log_dups_size":0,"ondisk_log_size":3461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14155776,"num_objects":3456,"num_object_clones":0,"num_object_copies":6912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":454,"num_write_kb":1816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"42'3405","reported_seq":3542,"reported_epoch":44,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:36.640351+0000","last_change":"2024-09-16T04:49:35.654625+0000","last_active":"2024-09-16T04:49:35.654625+0000","last_peered":"2024-09-16T04:49:35.625965+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:30.651379+0000","last_became_peered":"2024-09-16T04:49:30.651379+0000","last_unstale":"2024-09-16T04:49:36.640351+0000","last_undegraded":"2024-09-16T04:49:36.640351+0000","last_fullsized":"2024-09-16T04:49:36.640351+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":3405,"log_dups_size":0,"ondisk_log_size":3405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13922304,"num_objects":3399,"num_object_clones":0,"num_object_copies":6798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'3317","reported_seq":3526,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T04:49:36.641367+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:49:36.641367+0000","last_peered":"2024-09-16T04:49:36.641367+0000","last_clean":"2024-09-16T04:49:36.641367+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:36.641367+0000","last_undegraded":"2024-09-16T04:49:36.641367+0000","last_fullsized":"2024-09-16T04:49:36.641367+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":3317,"log_dups_size":0,"ondisk_log_size":3317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13553664,"num_objects":3309,"num_object_clones":0,"num_object_copies":6618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":387,"num_write_kb":1548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"42'3354","reported_seq":3569,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:35.601987+0000","last_change":"2024-09-16T04:49:35.649145+0000","last_active":"2024-09-16T04:49:35.649145+0000","last_peered":"2024-09-16T04:49:35.601987+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:30.606647+0000","last_became_peered":"2024-09-16T04:49:30.606647+0000","last_unstale":"2024-09-16T04:49:35.601987+0000","last_undegraded":"2024-09-16T04:49:35.601987+0000","last_fullsized":"2024-09-16T04:49:35.601987+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":3354,"log_dups_size":0,"ondisk_log_size":3354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13602816,"num_objects":3321,"num_object_clones":0,"num_object_copies":6642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-16T04:49:35.637733+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:35.637733+0000","last_peered":"2024-09-16T04:49:35.637733+0000","last_clean":"2024-09-16T04:49:35.637733+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:35.637733+0000","last_undegraded":"2024-09-16T04:49:35.637733+0000","last_fullsized":"2024-09-16T04:49:35.637733+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"42'3473","reported_seq":3592,"reported_epoch":44,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:36.649405+0000","last_change":"2024-09-16T04:49:35.657435+0000","last_active":"2024-09-16T04:49:35.657435+0000","last_peered":"2024-09-16T04:49:35.646646+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:30.601832+0000","last_became_peered":"2024-09-16T04:49:30.601832+0000","last_unstale":"2024-09-16T04:49:36.649405+0000","last_undegraded":"2024-09-16T04:49:36.649405+0000","last_fullsized":"2024-09-16T04:49:36.649405+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":3473,"log_dups_size":0,"ondisk_log_size":3473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14192640,"num_objects":3465,"num_object_clones":0,"num_object_copies":6930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":85,"reported_epoch":43,"state":"peering","last_fresh":"2024-09-16T04:49:35.659495+0000","last_change":"2024-09-16T04:49:36.649975+0000","last_active":"2024-09-16T04:49:35.659495+0000","last_peered":"2024-09-16T04:49:34.626115+0000","last_clean":"2024-09-16T04:49:34.626115+0000","last_became_active":"2024-09-16T04:49:30.643195+0000","last_became_peered":"2024-09-16T04:49:30.643195+0000","last_unstale":"2024-09-16T04:49:35.659495+0000","last_undegraded":"2024-09-16T04:49:35.659495+0000","last_fullsized":"2024-09-16T04:49:35.659495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"42'3327","reported_seq":3556,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:35.596121+0000","last_change":"2024-09-16T04:49:35.637542+0000","last_active":"2024-09-16T04:49:35.637542+0000","last_peered":"2024-09-16T04:49:35.596121+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:30.632699+0000","last_became_peered":"2024-09-16T04:49:30.632699+0000","last_unstale":"2024-09-16T04:49:35.596121+0000","last_undegraded":"2024-09-16T04:49:35.596121+0000","last_fullsized":"2024-09-16T04:49:35.596121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":3327,"log_dups_size":0,"ondisk_log_size":3327,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13501650,"num_objects":3298,"num_object_clones":0,"num_object_copies":6596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3299,"num_write_kb":13187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"42'3350","reported_seq":3494,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:35.635641+0000","last_change":"2024-09-16T04:49:35.636632+0000","last_active":"2024-09-16T04:49:35.636632+0000","last_peered":"2024-09-16T04:49:35.635641+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:30.607346+0000","last_became_peered":"2024-09-16T04:49:30.607346+0000","last_unstale":"2024-09-16T04:49:35.635641+0000","last_undegraded":"2024-09-16T04:49:35.635641+0000","last_fullsized":"2024-09-16T04:49:35.635641+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":3350,"log_dups_size":0,"ondisk_log_size":3350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13697024,"num_objects":3344,"num_object_clones":0,"num_object_copies":6688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"42'3409","reported_seq":3494,"reported_epoch":43,"state":"remapped","last_fresh":"2024-09-16T04:49:35.637782+0000","last_change":"2024-09-16T04:49:36.503444+0000","last_active":"2024-09-16T04:49:35.637782+0000","last_peered":"2024-09-16T04:49:35.635153+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:34.634801+0000","last_became_peered":"2024-09-16T04:49:34.634801+0000","last_unstale":"2024-09-16T04:49:35.637782+0000","last_undegraded":"2024-09-16T04:49:35.637782+0000","last_fullsized":"2024-09-16T04:49:35.637782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3409,"log_dups_size":0,"ondisk_log_size":3409,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13955689,"num_objects":3408,"num_object_clones":0,"num_object_copies":6816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3408,"num_write_kb":13629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"42'3522","reported_seq":3612,"reported_epoch":43,"state":"remapped","last_fresh":"2024-09-16T04:49:35.644896+0000","last_change":"2024-09-16T04:49:36.503404+0000","last_active":"2024-09-16T04:49:35.644896+0000","last_peered":"2024-09-16T04:49:35.584799+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:34.634723+0000","last_became_peered":"2024-09-16T04:49:34.634723+0000","last_unstale":"2024-09-16T04:49:35.644896+0000","last_undegraded":"2024-09-16T04:49:35.644896+0000","last_fullsized":"2024-09-16T04:49:35.644896+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14395195,"num_objects":3517,"num_object_clones":0,"num_object_copies":7034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3519,"num_write_kb":14060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-16T04:49:35.638581+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:35.638581+0000","last_peered":"2024-09-16T04:49:35.638581+0000","last_clean":"2024-09-16T04:49:35.638581+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:35.638581+0000","last_undegraded":"2024-09-16T04:49:35.638581+0000","last_fullsized":"2024-09-16T04:49:35.638581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"42'3329","reported_seq":3449,"reported_epoch":44,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:36.643103+0000","last_change":"2024-09-16T04:49:35.654562+0000","last_active":"2024-09-16T04:49:35.654562+0000","last_peered":"2024-09-16T04:49:35.635891+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:30.609068+0000","last_became_peered":"2024-09-16T04:49:30.609068+0000","last_unstale":"2024-09-16T04:49:36.643103+0000","last_undegraded":"2024-09-16T04:49:36.643103+0000","last_fullsized":"2024-09-16T04:49:36.643103+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":3329,"log_dups_size":0,"ondisk_log_size":3329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13619200,"num_objects":3325,"num_object_clones":0,"num_object_copies":6650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"42'3536","reported_seq":3665,"reported_epoch":44,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:36.639909+0000","last_change":"2024-09-16T04:49:35.653790+0000","last_active":"2024-09-16T04:49:35.653790+0000","last_peered":"2024-09-16T04:49:35.635773+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:30.614866+0000","last_became_peered":"2024-09-16T04:49:30.614866+0000","last_unstale":"2024-09-16T04:49:36.639909+0000","last_undegraded":"2024-09-16T04:49:36.639909+0000","last_fullsized":"2024-09-16T04:49:36.639909+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":3536,"log_dups_size":0,"ondisk_log_size":3536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14434921,"num_objects":3525,"num_object_clones":0,"num_object_copies":7050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":1220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":221720799,"num_objects":54137,"num_object_clones":0,"num_object_copies":108274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19371,"num_write_kb":77456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":12288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446648320,"data_stored":446561345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54293,"ondisk_log_size":54293,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334652,"kb_used_data":216868,"kb_used_omap":4,"kb_used_meta":117755,"kb_avail":94037188,"statfs":{"total":96636764160,"available":96294080512,"internally_reserved":0,"allocated":222072832,"data_stored":221823812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":120581801},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593730,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52512,"kb_used_data":2764,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94319328,"statfs":{"total":96636764160,"available":96582991872,"internally_reserved":0,"allocated":2830336,"data_stored":2495036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50917038},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":351240,"kb_used_data":216880,"kb_used_omap":4,"kb_used_meta":134331,"kb_avail":94020600,"statfs":{"total":96636764160,"available":96277094400,"internally_reserved":0,"allocated":222085120,"data_stored":221836100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":137555626},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,8,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":221720576,"data_stored":221717097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3219456,"data_stored":3139439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":221708288,"data_stored":221704809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:40.898 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:41.052 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 44 20 56046 56026 4.97314 2.65625 0.017249 0.0156957 2024-09-16T04:49:41.843 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:41.848 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:41.891 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-16T04:49:40.278223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":221720818,"num_objects":54139,"num_object_clones":0,"num_object_copies":108278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19373,"num_write_kb":77458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":12307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54296,"ondisk_log_size":54296,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":738404,"kb_used_data":436512,"kb_used_omap":12,"kb_used_meta":301811,"kb_avail":282377116,"statfs":{"total":289910292480,"available":289154166784,"internally_reserved":0,"allocated":446988288,"data_stored":446154948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":309054465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,6,14,6,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8144253,"num_objects":1979,"num_object_clones":0,"num_object_copies":3958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-39245,"num_objects_unfound":0,"num_objects_dirty":1979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-17696,"num_write_kb":-70707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-12,"num_bytes_recovered":-4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.653919"},"pg_stats":[{"pgid":"2.f","version":"42'3341","reported_seq":3409,"reported_epoch":43,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:35.654718+0000","last_change":"2024-09-16T04:49:35.654718+0000","last_active":"2024-09-16T04:49:35.654718+0000","last_peered":"2024-09-16T04:49:35.616010+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:30.661759+0000","last_became_peered":"2024-09-16T04:49:30.661759+0000","last_unstale":"2024-09-16T04:49:35.654718+0000","last_undegraded":"2024-09-16T04:49:35.654718+0000","last_fullsized":"2024-09-16T04:49:35.654718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3341,"log_dups_size":0,"ondisk_log_size":3341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13684736,"num_objects":3341,"num_object_clones":0,"num_object_copies":6682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3341,"num_write_kb":13364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"42'3362","reported_seq":3469,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-16T04:49:35.650619+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:49:35.650619+0000","last_peered":"2024-09-16T04:49:35.650619+0000","last_clean":"2024-09-16T04:49:35.650619+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:35.650619+0000","last_undegraded":"2024-09-16T04:49:35.650619+0000","last_fullsized":"2024-09-16T04:49:35.650619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":3362,"log_dups_size":0,"ondisk_log_size":3362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13746176,"num_objects":3356,"num_object_clones":0,"num_object_copies":6712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":331,"num_write_kb":1324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"42'3323","reported_seq":3448,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:35.615806+0000","last_change":"2024-09-16T04:49:35.644752+0000","last_active":"2024-09-16T04:49:35.644752+0000","last_peered":"2024-09-16T04:49:35.615806+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:30.606516+0000","last_became_peered":"2024-09-16T04:49:30.606516+0000","last_unstale":"2024-09-16T04:49:35.615806+0000","last_undegraded":"2024-09-16T04:49:35.615806+0000","last_fullsized":"2024-09-16T04:49:35.615806+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":3323,"log_dups_size":0,"ondisk_log_size":3323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13594624,"num_objects":3319,"num_object_clones":0,"num_object_copies":6638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'3397","reported_seq":3458,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-16T04:49:35.662330+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:35.662330+0000","last_peered":"2024-09-16T04:49:35.662330+0000","last_clean":"2024-09-16T04:49:35.662330+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:35.662330+0000","last_undegraded":"2024-09-16T04:49:35.662330+0000","last_fullsized":"2024-09-16T04:49:35.662330+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3397,"log_dups_size":0,"ondisk_log_size":3397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13914112,"num_objects":3397,"num_object_clones":0,"num_object_copies":6794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3397,"num_write_kb":13588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"42'3387","reported_seq":3599,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T04:49:36.640767+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:49:36.640767+0000","last_peered":"2024-09-16T04:49:36.640767+0000","last_clean":"2024-09-16T04:49:36.640767+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:36.640767+0000","last_undegraded":"2024-09-16T04:49:36.640767+0000","last_fullsized":"2024-09-16T04:49:36.640767+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":3387,"log_dups_size":0,"ondisk_log_size":3387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13750272,"num_objects":3357,"num_object_clones":0,"num_object_copies":6714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":419,"num_write_kb":1676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"42'3461","reported_seq":3583,"reported_epoch":44,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:36.641994+0000","last_change":"2024-09-16T04:49:35.657497+0000","last_active":"2024-09-16T04:49:35.657497+0000","last_peered":"2024-09-16T04:49:35.646925+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:30.638270+0000","last_became_peered":"2024-09-16T04:49:30.638270+0000","last_unstale":"2024-09-16T04:49:36.641994+0000","last_undegraded":"2024-09-16T04:49:36.641994+0000","last_fullsized":"2024-09-16T04:49:36.641994+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":3461,"log_dups_size":0,"ondisk_log_size":3461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14155776,"num_objects":3456,"num_object_clones":0,"num_object_copies":6912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":454,"num_write_kb":1816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"42'3405","reported_seq":3542,"reported_epoch":44,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:36.640351+0000","last_change":"2024-09-16T04:49:35.654625+0000","last_active":"2024-09-16T04:49:35.654625+0000","last_peered":"2024-09-16T04:49:35.625965+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:30.651379+0000","last_became_peered":"2024-09-16T04:49:30.651379+0000","last_unstale":"2024-09-16T04:49:36.640351+0000","last_undegraded":"2024-09-16T04:49:36.640351+0000","last_fullsized":"2024-09-16T04:49:36.640351+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":3405,"log_dups_size":0,"ondisk_log_size":3405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13922304,"num_objects":3399,"num_object_clones":0,"num_object_copies":6798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'3317","reported_seq":3526,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-16T04:49:36.641367+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:49:36.641367+0000","last_peered":"2024-09-16T04:49:36.641367+0000","last_clean":"2024-09-16T04:49:36.641367+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:36.641367+0000","last_undegraded":"2024-09-16T04:49:36.641367+0000","last_fullsized":"2024-09-16T04:49:36.641367+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":3317,"log_dups_size":0,"ondisk_log_size":3317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13553664,"num_objects":3309,"num_object_clones":0,"num_object_copies":6618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":387,"num_write_kb":1548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"42'3354","reported_seq":3569,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:35.601987+0000","last_change":"2024-09-16T04:49:35.649145+0000","last_active":"2024-09-16T04:49:35.649145+0000","last_peered":"2024-09-16T04:49:35.601987+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:30.606647+0000","last_became_peered":"2024-09-16T04:49:30.606647+0000","last_unstale":"2024-09-16T04:49:35.601987+0000","last_undegraded":"2024-09-16T04:49:35.601987+0000","last_fullsized":"2024-09-16T04:49:35.601987+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":3354,"log_dups_size":0,"ondisk_log_size":3354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13602816,"num_objects":3321,"num_object_clones":0,"num_object_copies":6642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-16T04:49:35.637733+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:35.637733+0000","last_peered":"2024-09-16T04:49:35.637733+0000","last_clean":"2024-09-16T04:49:35.637733+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:35.637733+0000","last_undegraded":"2024-09-16T04:49:35.637733+0000","last_fullsized":"2024-09-16T04:49:35.637733+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"42'3473","reported_seq":3592,"reported_epoch":44,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:36.649405+0000","last_change":"2024-09-16T04:49:35.657435+0000","last_active":"2024-09-16T04:49:35.657435+0000","last_peered":"2024-09-16T04:49:35.646646+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:30.601832+0000","last_became_peered":"2024-09-16T04:49:30.601832+0000","last_unstale":"2024-09-16T04:49:36.649405+0000","last_undegraded":"2024-09-16T04:49:36.649405+0000","last_fullsized":"2024-09-16T04:49:36.649405+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":3473,"log_dups_size":0,"ondisk_log_size":3473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14192640,"num_objects":3465,"num_object_clones":0,"num_object_copies":6930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":85,"reported_epoch":43,"state":"peering","last_fresh":"2024-09-16T04:49:35.659495+0000","last_change":"2024-09-16T04:49:36.649975+0000","last_active":"2024-09-16T04:49:35.659495+0000","last_peered":"2024-09-16T04:49:34.626115+0000","last_clean":"2024-09-16T04:49:34.626115+0000","last_became_active":"2024-09-16T04:49:30.643195+0000","last_became_peered":"2024-09-16T04:49:30.643195+0000","last_unstale":"2024-09-16T04:49:35.659495+0000","last_undegraded":"2024-09-16T04:49:35.659495+0000","last_fullsized":"2024-09-16T04:49:35.659495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"42'3327","reported_seq":3556,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:35.596121+0000","last_change":"2024-09-16T04:49:35.637542+0000","last_active":"2024-09-16T04:49:35.637542+0000","last_peered":"2024-09-16T04:49:35.596121+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:30.632699+0000","last_became_peered":"2024-09-16T04:49:30.632699+0000","last_unstale":"2024-09-16T04:49:35.596121+0000","last_undegraded":"2024-09-16T04:49:35.596121+0000","last_fullsized":"2024-09-16T04:49:35.596121+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":3327,"log_dups_size":0,"ondisk_log_size":3327,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13501650,"num_objects":3298,"num_object_clones":0,"num_object_copies":6596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3299,"num_write_kb":13187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"42'3350","reported_seq":3494,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:35.635641+0000","last_change":"2024-09-16T04:49:35.636632+0000","last_active":"2024-09-16T04:49:35.636632+0000","last_peered":"2024-09-16T04:49:35.635641+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:30.607346+0000","last_became_peered":"2024-09-16T04:49:30.607346+0000","last_unstale":"2024-09-16T04:49:35.635641+0000","last_undegraded":"2024-09-16T04:49:35.635641+0000","last_fullsized":"2024-09-16T04:49:35.635641+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":3350,"log_dups_size":0,"ondisk_log_size":3350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13697024,"num_objects":3344,"num_object_clones":0,"num_object_copies":6688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"42'3409","reported_seq":3494,"reported_epoch":43,"state":"remapped","last_fresh":"2024-09-16T04:49:35.637782+0000","last_change":"2024-09-16T04:49:36.503444+0000","last_active":"2024-09-16T04:49:35.637782+0000","last_peered":"2024-09-16T04:49:35.635153+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:34.634801+0000","last_became_peered":"2024-09-16T04:49:34.634801+0000","last_unstale":"2024-09-16T04:49:35.637782+0000","last_undegraded":"2024-09-16T04:49:35.637782+0000","last_fullsized":"2024-09-16T04:49:35.637782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3409,"log_dups_size":0,"ondisk_log_size":3409,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13955689,"num_objects":3408,"num_object_clones":0,"num_object_copies":6816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3408,"num_write_kb":13629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"42'3522","reported_seq":3612,"reported_epoch":43,"state":"remapped","last_fresh":"2024-09-16T04:49:35.644896+0000","last_change":"2024-09-16T04:49:36.503404+0000","last_active":"2024-09-16T04:49:35.644896+0000","last_peered":"2024-09-16T04:49:35.584799+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:34.634723+0000","last_became_peered":"2024-09-16T04:49:34.634723+0000","last_unstale":"2024-09-16T04:49:35.644896+0000","last_undegraded":"2024-09-16T04:49:35.644896+0000","last_fullsized":"2024-09-16T04:49:35.644896+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14395195,"num_objects":3517,"num_object_clones":0,"num_object_copies":7034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3519,"num_write_kb":14060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-16T04:49:35.638581+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:35.638581+0000","last_peered":"2024-09-16T04:49:35.638581+0000","last_clean":"2024-09-16T04:49:35.638581+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:35.638581+0000","last_undegraded":"2024-09-16T04:49:35.638581+0000","last_fullsized":"2024-09-16T04:49:35.638581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"42'3329","reported_seq":3449,"reported_epoch":44,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:36.643103+0000","last_change":"2024-09-16T04:49:35.654562+0000","last_active":"2024-09-16T04:49:35.654562+0000","last_peered":"2024-09-16T04:49:35.635891+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:30.609068+0000","last_became_peered":"2024-09-16T04:49:30.609068+0000","last_unstale":"2024-09-16T04:49:36.643103+0000","last_undegraded":"2024-09-16T04:49:36.643103+0000","last_fullsized":"2024-09-16T04:49:36.643103+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":3329,"log_dups_size":0,"ondisk_log_size":3329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13619200,"num_objects":3325,"num_object_clones":0,"num_object_copies":6650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"42'3536","reported_seq":3665,"reported_epoch":44,"state":"remapped+peering","last_fresh":"2024-09-16T04:49:36.639909+0000","last_change":"2024-09-16T04:49:35.653790+0000","last_active":"2024-09-16T04:49:35.653790+0000","last_peered":"2024-09-16T04:49:35.635773+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:30.614866+0000","last_became_peered":"2024-09-16T04:49:30.614866+0000","last_unstale":"2024-09-16T04:49:36.639909+0000","last_undegraded":"2024-09-16T04:49:36.639909+0000","last_fullsized":"2024-09-16T04:49:36.639909+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":3536,"log_dups_size":0,"ondisk_log_size":3536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14434921,"num_objects":3525,"num_object_clones":0,"num_object_copies":7050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":1220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":221720799,"num_objects":54137,"num_object_clones":0,"num_object_copies":108274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19371,"num_write_kb":77456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":12288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446648320,"data_stored":446561345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54293,"ondisk_log_size":54293,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":334652,"kb_used_data":216868,"kb_used_omap":4,"kb_used_meta":117755,"kb_avail":94037188,"statfs":{"total":96636764160,"available":96294080512,"internally_reserved":0,"allocated":222072832,"data_stored":221823812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":120581801},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593730,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52512,"kb_used_data":2764,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94319328,"statfs":{"total":96636764160,"available":96582991872,"internally_reserved":0,"allocated":2830336,"data_stored":2495036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50917038},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":351240,"kb_used_data":216880,"kb_used_omap":4,"kb_used_meta":134331,"kb_avail":94020600,"statfs":{"total":96636764160,"available":96277094400,"internally_reserved":0,"allocated":222085120,"data_stored":221836100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":137555626},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,8,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":221720576,"data_stored":221717097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3219456,"data_stored":3139439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":221708288,"data_stored":221704809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:41.892 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:42.053 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 45 20 56743 56723 4.923 2.72266 0.0501137 0.0158586 2024-09-16T04:49:42.741 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:42.741 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:42.800 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-16T04:49:42.278587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":231747826,"num_objects":56587,"num_object_clones":0,"num_object_copies":113174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3272,"num_objects_unfound":0,"num_objects_dirty":56587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21821,"num_write_kb":87250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":307219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56744,"ondisk_log_size":56744,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":742040,"kb_used_data":448484,"kb_used_omap":12,"kb_used_meta":293491,"kb_avail":282373480,"statfs":{"total":289910292480,"available":289150443520,"internally_reserved":0,"allocated":459247616,"data_stored":458394792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":300534785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,4,7,7,1,2,1,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18171261,"num_objects":4427,"num_object_clones":0,"num_object_copies":8854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":-35973,"num_objects_unfound":0,"num_objects_dirty":4427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-15248,"num_write_kb":-60915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":290816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.657508"},"pg_stats":[{"pgid":"2.f","version":"45'3503","reported_seq":3575,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:42.105707+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:49:42.105707+0000","last_peered":"2024-09-16T04:49:42.105707+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:49:42.105707+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3503,"log_dups_size":0,"ondisk_log_size":3503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14348288,"num_objects":3503,"num_object_clones":0,"num_object_copies":7006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3503,"num_write_kb":14012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":162},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'3516","reported_seq":3627,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:41.498646+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:49:41.498646+0000","last_peered":"2024-09-16T04:49:41.498646+0000","last_clean":"2024-09-16T04:49:41.498646+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:41.498646+0000","last_undegraded":"2024-09-16T04:49:41.498646+0000","last_fullsized":"2024-09-16T04:49:41.498646+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":3516,"log_dups_size":0,"ondisk_log_size":3516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14376960,"num_objects":3510,"num_object_clones":0,"num_object_copies":7020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":485,"num_write_kb":1940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'3476","reported_seq":3608,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:41.480086+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:49:41.480086+0000","last_peered":"2024-09-16T04:49:41.480086+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:49:41.480086+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":3476,"log_dups_size":0,"ondisk_log_size":3476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14221312,"num_objects":3472,"num_object_clones":0,"num_object_copies":6944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":215,"num_write_kb":860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":153}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'3544","reported_seq":3609,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:41.422453+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:41.422453+0000","last_peered":"2024-09-16T04:49:41.422453+0000","last_clean":"2024-09-16T04:49:41.422453+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:41.422453+0000","last_undegraded":"2024-09-16T04:49:41.422453+0000","last_fullsized":"2024-09-16T04:49:41.422453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3544,"log_dups_size":0,"ondisk_log_size":3544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14516224,"num_objects":3544,"num_object_clones":0,"num_object_copies":7088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3544,"num_write_kb":14176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'3555","reported_seq":3769,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:41.999441+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:49:41.999441+0000","last_peered":"2024-09-16T04:49:41.999441+0000","last_clean":"2024-09-16T04:49:41.999441+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:41.999441+0000","last_undegraded":"2024-09-16T04:49:41.999441+0000","last_fullsized":"2024-09-16T04:49:41.999441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":3555,"log_dups_size":0,"ondisk_log_size":3555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14438400,"num_objects":3525,"num_object_clones":0,"num_object_copies":7050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":587,"num_write_kb":2348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'3603","reported_seq":3730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:42.089230+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:49:42.089230+0000","last_peered":"2024-09-16T04:49:42.089230+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:49:42.089230+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":3603,"log_dups_size":0,"ondisk_log_size":3603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14737408,"num_objects":3598,"num_object_clones":0,"num_object_copies":7196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":596,"num_write_kb":2384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":142},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'3563","reported_seq":3703,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:42.055123+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:49:42.055123+0000","last_peered":"2024-09-16T04:49:42.055123+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:49:42.055123+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":3563,"log_dups_size":0,"ondisk_log_size":3563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14569472,"num_objects":3557,"num_object_clones":0,"num_object_copies":7114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":218,"num_write_kb":872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":158},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'3483","reported_seq":3693,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:42.106078+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:49:42.106078+0000","last_peered":"2024-09-16T04:49:42.106078+0000","last_clean":"2024-09-16T04:49:42.106078+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:42.106078+0000","last_undegraded":"2024-09-16T04:49:42.106078+0000","last_fullsized":"2024-09-16T04:49:42.106078+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":3483,"log_dups_size":0,"ondisk_log_size":3483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14233600,"num_objects":3475,"num_object_clones":0,"num_object_copies":6950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":553,"num_write_kb":2212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'3476","reported_seq":3699,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:41.465183+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:49:41.465183+0000","last_peered":"2024-09-16T04:49:41.465183+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:49:41.465183+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":3476,"log_dups_size":0,"ondisk_log_size":3476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14102528,"num_objects":3443,"num_object_clones":0,"num_object_copies":6886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":227,"num_write_kb":908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'3647","reported_seq":3771,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:42.093897+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:49:42.093897+0000","last_peered":"2024-09-16T04:49:42.093897+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:49:42.093897+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":3647,"log_dups_size":0,"ondisk_log_size":3647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14905344,"num_objects":3639,"num_object_clones":0,"num_object_copies":7278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":302,"num_write_kb":1208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":174},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'3474","reported_seq":3711,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:41.482084+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:49:41.482084+0000","last_peered":"2024-09-16T04:49:41.482084+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:49:41.482084+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":3474,"log_dups_size":0,"ondisk_log_size":3474,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14103762,"num_objects":3445,"num_object_clones":0,"num_object_copies":6890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3446,"num_write_kb":13775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":149}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'3502","reported_seq":3799,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:49:41.474084+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:49:41.474084+0000","last_peered":"2024-09-16T04:49:41.474084+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:49:41.474084+0000","last_undegraded":"2024-09-16T04:49:41.474084+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":3502,"log_dups_size":0,"ondisk_log_size":3502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14319616,"num_objects":3496,"num_object_clones":0,"num_object_copies":6992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3272,"num_objects_unfound":0,"num_objects_dirty":3496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":225,"num_write_kb":900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":294912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3272},{"shards":"2","objects":224}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'3553","reported_seq":3646,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:41.481667+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:49:41.481667+0000","last_peered":"2024-09-16T04:49:41.481667+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:49:41.481667+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3553,"log_dups_size":0,"ondisk_log_size":3553,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14545513,"num_objects":3552,"num_object_clones":0,"num_object_copies":7104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3552,"num_write_kb":14205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":145}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'3665","reported_seq":3763,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:41.485474+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:49:41.485474+0000","last_peered":"2024-09-16T04:49:41.485474+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:49:41.485474+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":3665,"log_dups_size":0,"ondisk_log_size":3665,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14980923,"num_objects":3660,"num_object_clones":0,"num_object_copies":7320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3662,"num_write_kb":14632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":146}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'3483","reported_seq":3609,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:42.096737+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:49:42.096737+0000","last_peered":"2024-09-16T04:49:42.096737+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:49:42.096737+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":3483,"log_dups_size":0,"ondisk_log_size":3483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14249984,"num_objects":3479,"num_object_clones":0,"num_object_copies":6958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":237,"num_write_kb":948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":154},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'3698","reported_seq":3833,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:42.089256+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:49:42.089256+0000","last_peered":"2024-09-16T04:49:42.089256+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:49:42.089256+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":3698,"log_dups_size":0,"ondisk_log_size":3698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15098473,"num_objects":3687,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":1868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":163},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":231747807,"num_objects":56585,"num_object_clones":0,"num_object_copies":113170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3272,"num_objects_unfound":0,"num_objects_dirty":56585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21819,"num_write_kb":87248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":307200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":459251712,"data_stored":459164737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56741,"ondisk_log_size":56741,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":344200,"kb_used_data":222264,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94027640,"statfs":{"total":96636764160,"available":96284303360,"internally_reserved":0,"allocated":227598336,"data_stored":227339574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":124841641},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,6,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593730,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52512,"kb_used_data":2764,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94319328,"statfs":{"total":96636764160,"available":96582991872,"internally_reserved":0,"allocated":2830336,"data_stored":2495036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50917038},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":345328,"kb_used_data":223456,"kb_used_omap":4,"kb_used_meta":121851,"kb_avail":94026512,"statfs":{"total":96636764160,"available":96283148288,"internally_reserved":0,"allocated":228818944,"data_stored":228560182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":124776106},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1,0,2,1,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":228446208,"data_stored":228442729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3219456,"data_stored":3139439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":227586048,"data_stored":227582569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:42.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:49:42.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'3503", 'reported_seq': 3575, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:42.105707+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:49:42.105707+0000', 'last_peered': '2024-09-16T04:49:42.105707+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:49:42.105707+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 3503, 'log_dups_size': 0, 'ondisk_log_size': 3503, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14348288, 'num_objects': 3503, 'num_object_clones': 0, 'num_object_copies': 7006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3503, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3503, 'num_write_kb': 14012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 162}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:42.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:49:42.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'3476", 'reported_seq': 3608, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:41.480086+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:49:41.480086+0000', 'last_peered': '2024-09-16T04:49:41.480086+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:49:41.480086+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 3476, 'log_dups_size': 0, 'ondisk_log_size': 3476, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14221312, 'num_objects': 3472, 'num_object_clones': 0, 'num_object_copies': 6944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 215, 'num_write_kb': 860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 153}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:42.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:49:42.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'3603", 'reported_seq': 3730, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:42.089230+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:49:42.089230+0000', 'last_peered': '2024-09-16T04:49:42.089230+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:49:42.089230+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 3603, 'log_dups_size': 0, 'ondisk_log_size': 3603, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14737408, 'num_objects': 3598, 'num_object_clones': 0, 'num_object_copies': 7196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3598, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 596, 'num_write_kb': 2384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 142}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:42.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:49:42.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'3563", 'reported_seq': 3703, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:42.055123+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:49:42.055123+0000', 'last_peered': '2024-09-16T04:49:42.055123+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:49:42.055123+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 3563, 'log_dups_size': 0, 'ondisk_log_size': 3563, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14569472, 'num_objects': 3557, 'num_object_clones': 0, 'num_object_copies': 7114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3557, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 218, 'num_write_kb': 872, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 158}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:42.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:49:42.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'3476", 'reported_seq': 3699, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:41.465183+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:49:41.465183+0000', 'last_peered': '2024-09-16T04:49:41.465183+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:49:41.465183+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 3476, 'log_dups_size': 0, 'ondisk_log_size': 3476, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14102528, 'num_objects': 3443, 'num_object_clones': 0, 'num_object_copies': 6886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3443, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 227, 'num_write_kb': 908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 122}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:42.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:49:42.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'3647", 'reported_seq': 3771, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:42.093897+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:49:42.093897+0000', 'last_peered': '2024-09-16T04:49:42.093897+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:49:42.093897+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 3647, 'log_dups_size': 0, 'ondisk_log_size': 3647, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14905344, 'num_objects': 3639, 'num_object_clones': 0, 'num_object_copies': 7278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3639, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 302, 'num_write_kb': 1208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 174}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:42.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:49:42.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'3474", 'reported_seq': 3711, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:41.482084+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:49:41.482084+0000', 'last_peered': '2024-09-16T04:49:41.482084+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:49:41.482084+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 3474, 'log_dups_size': 0, 'ondisk_log_size': 3474, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14103762, 'num_objects': 3445, 'num_object_clones': 0, 'num_object_copies': 6890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3445, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3446, 'num_write_kb': 13775, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 149}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:42.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:49:42.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'3502", 'reported_seq': 3799, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:49:41.474084+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:49:41.474084+0000', 'last_peered': '2024-09-16T04:49:41.474084+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:49:41.474084+0000', 'last_undegraded': '2024-09-16T04:49:41.474084+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 3502, 'log_dups_size': 0, 'ondisk_log_size': 3502, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14319616, 'num_objects': 3496, 'num_object_clones': 0, 'num_object_copies': 6992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3272, 'num_objects_unfound': 0, 'num_objects_dirty': 3496, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 225, 'num_write_kb': 900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 72, 'num_bytes_recovered': 294912, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3272}, {'shards': '2', 'objects': 224}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:42.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:49:42.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'3553", 'reported_seq': 3646, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:41.481667+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:49:41.481667+0000', 'last_peered': '2024-09-16T04:49:41.481667+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:49:41.481667+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 3553, 'log_dups_size': 0, 'ondisk_log_size': 3553, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14545513, 'num_objects': 3552, 'num_object_clones': 0, 'num_object_copies': 7104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3552, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3552, 'num_write_kb': 14205, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 145}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:42.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:49:42.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'3665", 'reported_seq': 3763, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:41.485474+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:49:41.485474+0000', 'last_peered': '2024-09-16T04:49:41.485474+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:49:41.485474+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 3665, 'log_dups_size': 0, 'ondisk_log_size': 3665, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14980923, 'num_objects': 3660, 'num_object_clones': 0, 'num_object_copies': 7320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3660, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3662, 'num_write_kb': 14632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 146}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:42.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:49:42.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'3483", 'reported_seq': 3609, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:42.096737+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:49:42.096737+0000', 'last_peered': '2024-09-16T04:49:42.096737+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:49:42.096737+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 3483, 'log_dups_size': 0, 'ondisk_log_size': 3483, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14249984, 'num_objects': 3479, 'num_object_clones': 0, 'num_object_copies': 6958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3479, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 237, 'num_write_kb': 948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 154}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:42.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:49:42.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'3698", 'reported_seq': 3833, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:42.089256+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:49:42.089256+0000', 'last_peered': '2024-09-16T04:49:42.089256+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:49:42.089256+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 3698, 'log_dups_size': 0, 'ondisk_log_size': 3698, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15098473, 'num_objects': 3687, 'num_object_clones': 0, 'num_object_copies': 7374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3687, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 467, 'num_write_kb': 1868, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 163}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:42.804 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:49:43.053 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 46 20 57404 57384 4.8721 2.58203 0.0870929 0.0160231 2024-09-16T04:49:43.985 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:43.986 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/113174 objects degraded (33.035%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":131,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":56587,"data_bytes":231747826,"bytes_used":759848960,"bytes_avail":289150443520,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":113174,"degraded_ratio":0.33034972696909182,"misplaced_objects":3272,"misplaced_total":113174,"misplaced_ratio":0.028911234029017265,"recovering_objects_per_sec":9,"recovering_bytes_per_sec":43682,"recovering_keys_per_sec":0,"num_objects_recovered":60,"num_bytes_recovered":290816,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-16T04:49:42.278701+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T04:49:44.053 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 47 20 58072 58052 4.82396 2.60938 0.0301648 0.016183 2024-09-16T04:49:44.071 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:49:44.072 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:44.963 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:44.964 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:45.029 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-16T04:49:44.279167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":231747826,"num_objects":56587,"num_object_clones":0,"num_object_copies":113174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3272,"num_objects_unfound":0,"num_objects_dirty":56587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21821,"num_write_kb":87250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":307219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56744,"ondisk_log_size":56744,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":742040,"kb_used_data":448484,"kb_used_omap":12,"kb_used_meta":293491,"kb_avail":282373480,"statfs":{"total":289910292480,"available":289150443520,"internally_reserved":0,"allocated":459247616,"data_stored":458394792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":300534785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,4,7,7,1,2,1,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18171261,"num_objects":4427,"num_object_clones":0,"num_object_copies":8854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":-35973,"num_objects_unfound":0,"num_objects_dirty":4427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-15248,"num_write_kb":-60915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":290816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002725"},"pg_stats":[{"pgid":"2.f","version":"45'3503","reported_seq":3575,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:42.105707+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:49:42.105707+0000","last_peered":"2024-09-16T04:49:42.105707+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:49:42.105707+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3503,"log_dups_size":0,"ondisk_log_size":3503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14348288,"num_objects":3503,"num_object_clones":0,"num_object_copies":7006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3503,"num_write_kb":14012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":162},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'3516","reported_seq":3627,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:41.498646+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:49:41.498646+0000","last_peered":"2024-09-16T04:49:41.498646+0000","last_clean":"2024-09-16T04:49:41.498646+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:41.498646+0000","last_undegraded":"2024-09-16T04:49:41.498646+0000","last_fullsized":"2024-09-16T04:49:41.498646+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":3516,"log_dups_size":0,"ondisk_log_size":3516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14376960,"num_objects":3510,"num_object_clones":0,"num_object_copies":7020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":485,"num_write_kb":1940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'3476","reported_seq":3608,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:41.480086+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:49:41.480086+0000","last_peered":"2024-09-16T04:49:41.480086+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:49:41.480086+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":3476,"log_dups_size":0,"ondisk_log_size":3476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14221312,"num_objects":3472,"num_object_clones":0,"num_object_copies":6944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":215,"num_write_kb":860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":153}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'3544","reported_seq":3609,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:41.422453+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:41.422453+0000","last_peered":"2024-09-16T04:49:41.422453+0000","last_clean":"2024-09-16T04:49:41.422453+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:41.422453+0000","last_undegraded":"2024-09-16T04:49:41.422453+0000","last_fullsized":"2024-09-16T04:49:41.422453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3544,"log_dups_size":0,"ondisk_log_size":3544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14516224,"num_objects":3544,"num_object_clones":0,"num_object_copies":7088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3544,"num_write_kb":14176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'3555","reported_seq":3769,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:41.999441+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:49:41.999441+0000","last_peered":"2024-09-16T04:49:41.999441+0000","last_clean":"2024-09-16T04:49:41.999441+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:41.999441+0000","last_undegraded":"2024-09-16T04:49:41.999441+0000","last_fullsized":"2024-09-16T04:49:41.999441+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":3555,"log_dups_size":0,"ondisk_log_size":3555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14438400,"num_objects":3525,"num_object_clones":0,"num_object_copies":7050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":587,"num_write_kb":2348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'3603","reported_seq":3730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:42.089230+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:49:42.089230+0000","last_peered":"2024-09-16T04:49:42.089230+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:49:42.089230+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":3603,"log_dups_size":0,"ondisk_log_size":3603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14737408,"num_objects":3598,"num_object_clones":0,"num_object_copies":7196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":596,"num_write_kb":2384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":142},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'3563","reported_seq":3703,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:42.055123+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:49:42.055123+0000","last_peered":"2024-09-16T04:49:42.055123+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:49:42.055123+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":3563,"log_dups_size":0,"ondisk_log_size":3563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14569472,"num_objects":3557,"num_object_clones":0,"num_object_copies":7114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":218,"num_write_kb":872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":158},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'3483","reported_seq":3693,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:42.106078+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:49:42.106078+0000","last_peered":"2024-09-16T04:49:42.106078+0000","last_clean":"2024-09-16T04:49:42.106078+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:42.106078+0000","last_undegraded":"2024-09-16T04:49:42.106078+0000","last_fullsized":"2024-09-16T04:49:42.106078+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":3483,"log_dups_size":0,"ondisk_log_size":3483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14233600,"num_objects":3475,"num_object_clones":0,"num_object_copies":6950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":553,"num_write_kb":2212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'3476","reported_seq":3699,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:41.465183+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:49:41.465183+0000","last_peered":"2024-09-16T04:49:41.465183+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:49:41.465183+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":3476,"log_dups_size":0,"ondisk_log_size":3476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14102528,"num_objects":3443,"num_object_clones":0,"num_object_copies":6886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":227,"num_write_kb":908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'3647","reported_seq":3771,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:42.093897+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:49:42.093897+0000","last_peered":"2024-09-16T04:49:42.093897+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:49:42.093897+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":3647,"log_dups_size":0,"ondisk_log_size":3647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14905344,"num_objects":3639,"num_object_clones":0,"num_object_copies":7278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":302,"num_write_kb":1208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":174},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'3474","reported_seq":3711,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:41.482084+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:49:41.482084+0000","last_peered":"2024-09-16T04:49:41.482084+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:49:41.482084+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":3474,"log_dups_size":0,"ondisk_log_size":3474,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14103762,"num_objects":3445,"num_object_clones":0,"num_object_copies":6890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3446,"num_write_kb":13775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":149}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'3502","reported_seq":3799,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:49:41.474084+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:49:41.474084+0000","last_peered":"2024-09-16T04:49:41.474084+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:49:41.474084+0000","last_undegraded":"2024-09-16T04:49:41.474084+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":3502,"log_dups_size":0,"ondisk_log_size":3502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14319616,"num_objects":3496,"num_object_clones":0,"num_object_copies":6992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3272,"num_objects_unfound":0,"num_objects_dirty":3496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":225,"num_write_kb":900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":294912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3272},{"shards":"2","objects":224}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'3553","reported_seq":3646,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:41.481667+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:49:41.481667+0000","last_peered":"2024-09-16T04:49:41.481667+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:49:41.481667+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3553,"log_dups_size":0,"ondisk_log_size":3553,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14545513,"num_objects":3552,"num_object_clones":0,"num_object_copies":7104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3552,"num_write_kb":14205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":145}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'3665","reported_seq":3763,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:41.485474+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:49:41.485474+0000","last_peered":"2024-09-16T04:49:41.485474+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:49:41.485474+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":3665,"log_dups_size":0,"ondisk_log_size":3665,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14980923,"num_objects":3660,"num_object_clones":0,"num_object_copies":7320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3662,"num_write_kb":14632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":146}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'3483","reported_seq":3609,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:42.096737+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:49:42.096737+0000","last_peered":"2024-09-16T04:49:42.096737+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:49:42.096737+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":3483,"log_dups_size":0,"ondisk_log_size":3483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14249984,"num_objects":3479,"num_object_clones":0,"num_object_copies":6958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":237,"num_write_kb":948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":154},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'3698","reported_seq":3833,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:42.089256+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:49:42.089256+0000","last_peered":"2024-09-16T04:49:42.089256+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:49:42.089256+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":3698,"log_dups_size":0,"ondisk_log_size":3698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15098473,"num_objects":3687,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":1868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":163},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":231747807,"num_objects":56585,"num_object_clones":0,"num_object_copies":113170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3272,"num_objects_unfound":0,"num_objects_dirty":56585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21819,"num_write_kb":87248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":307200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":459251712,"data_stored":459164737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56741,"ondisk_log_size":56741,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":344200,"kb_used_data":222264,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94027640,"statfs":{"total":96636764160,"available":96284303360,"internally_reserved":0,"allocated":227598336,"data_stored":227339574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":124841641},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,6,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593730,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52512,"kb_used_data":2764,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94319328,"statfs":{"total":96636764160,"available":96582991872,"internally_reserved":0,"allocated":2830336,"data_stored":2495036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50917038},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":345328,"kb_used_data":223456,"kb_used_omap":4,"kb_used_meta":121851,"kb_avail":94026512,"statfs":{"total":96636764160,"available":96283148288,"internally_reserved":0,"allocated":228818944,"data_stored":228560182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":124776106},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1,0,2,1,0,0,0,6],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":228446208,"data_stored":228442729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3219456,"data_stored":3139439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":227586048,"data_stored":227582569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:45.054 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 48 20 58738 58718 4.77763 2.60156 0.0232425 0.0163433 2024-09-16T04:49:46.054 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 49 20 59463 59443 4.73793 2.83203 0.0368557 0.0164814 2024-09-16T04:49:47.054 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 50 20 60166 60146 4.69809 2.74609 0.00752724 0.0166188 2024-09-16T04:49:48.031 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:48.054 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 51 20 60878 60858 4.6605 2.78125 0.0196119 0.0167552 2024-09-16T04:49:48.888 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:48.889 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:48.931 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-16T04:49:48.280443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":245711090,"num_objects":59996,"num_object_clones":0,"num_object_copies":119992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3161,"num_objects_unfound":0,"num_objects_dirty":59996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25230,"num_write_kb":100886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":761875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60153,"ondisk_log_size":60153,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":826716,"kb_used_data":476268,"kb_used_omap":12,"kb_used_meta":350387,"kb_avail":282288804,"statfs":{"total":289910292480,"available":289063735296,"internally_reserved":0,"allocated":487698432,"data_stored":486845608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":358796289},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,3,4,8,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23990272,"num_objects":5857,"num_object_clones":0,"num_object_copies":11714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3161,"num_objects_unfound":0,"num_objects_dirty":5857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5857,"num_write_kb":23428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":749568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.643278"},"pg_stats":[{"pgid":"2.f","version":"45'3738","reported_seq":3810,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.099662+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:49:47.099662+0000","last_peered":"2024-09-16T04:49:47.099662+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:49:47.099662+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3738,"log_dups_size":0,"ondisk_log_size":3738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15310848,"num_objects":3738,"num_object_clones":0,"num_object_copies":7476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3738,"num_write_kb":14952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":397},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'3755","reported_seq":3866,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:46.483882+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:49:46.483882+0000","last_peered":"2024-09-16T04:49:46.483882+0000","last_clean":"2024-09-16T04:49:46.483882+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:46.483882+0000","last_undegraded":"2024-09-16T04:49:46.483882+0000","last_fullsized":"2024-09-16T04:49:46.483882+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":3755,"log_dups_size":0,"ondisk_log_size":3755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15355904,"num_objects":3749,"num_object_clones":0,"num_object_copies":7498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":724,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'3676","reported_seq":3809,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.491100+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:49:46.491100+0000","last_peered":"2024-09-16T04:49:46.491100+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:49:46.491100+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":3676,"log_dups_size":0,"ondisk_log_size":3676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15040512,"num_objects":3672,"num_object_clones":0,"num_object_copies":7344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":415,"num_write_kb":1660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":353}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'3743","reported_seq":3808,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:46.414317+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:46.414317+0000","last_peered":"2024-09-16T04:49:46.414317+0000","last_clean":"2024-09-16T04:49:46.414317+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:46.414317+0000","last_undegraded":"2024-09-16T04:49:46.414317+0000","last_fullsized":"2024-09-16T04:49:46.414317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15331328,"num_objects":3743,"num_object_clones":0,"num_object_copies":7486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3743,"num_write_kb":14972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'3755","reported_seq":3969,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:47.073660+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:49:47.073660+0000","last_peered":"2024-09-16T04:49:47.073660+0000","last_clean":"2024-09-16T04:49:47.073660+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:47.073660+0000","last_undegraded":"2024-09-16T04:49:47.073660+0000","last_fullsized":"2024-09-16T04:49:47.073660+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":3755,"log_dups_size":0,"ondisk_log_size":3755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15257600,"num_objects":3725,"num_object_clones":0,"num_object_copies":7450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":787,"num_write_kb":3148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'3816","reported_seq":3944,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.042800+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:49:47.042800+0000","last_peered":"2024-09-16T04:49:47.042800+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:49:47.042800+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":3816,"log_dups_size":0,"ondisk_log_size":3816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15609856,"num_objects":3811,"num_object_clones":0,"num_object_copies":7622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":809,"num_write_kb":3236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":355},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'3768","reported_seq":3911,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.034518+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:49:47.034518+0000","last_peered":"2024-09-16T04:49:47.034518+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:49:47.034518+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":3768,"log_dups_size":0,"ondisk_log_size":3768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15409152,"num_objects":3762,"num_object_clones":0,"num_object_copies":7524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":423,"num_write_kb":1692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":363},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'3691","reported_seq":3902,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:47.067403+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:49:47.067403+0000","last_peered":"2024-09-16T04:49:47.067403+0000","last_clean":"2024-09-16T04:49:47.067403+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:47.067403+0000","last_undegraded":"2024-09-16T04:49:47.067403+0000","last_fullsized":"2024-09-16T04:49:47.067403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":3691,"log_dups_size":0,"ondisk_log_size":3691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15085568,"num_objects":3683,"num_object_clones":0,"num_object_copies":7366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":761,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'3683","reported_seq":3906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.475890+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:49:46.475890+0000","last_peered":"2024-09-16T04:49:46.475890+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:49:46.475890+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":3683,"log_dups_size":0,"ondisk_log_size":3683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14950400,"num_objects":3650,"num_object_clones":0,"num_object_copies":7300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":434,"num_write_kb":1736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":329}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'3847","reported_seq":3972,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.102435+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:49:47.102435+0000","last_peered":"2024-09-16T04:49:47.102435+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:49:47.102435+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":3847,"log_dups_size":0,"ondisk_log_size":3847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15724544,"num_objects":3839,"num_object_clones":0,"num_object_copies":7678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":502,"num_write_kb":2008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":374},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'3710","reported_seq":3947,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.496863+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:49:46.496863+0000","last_peered":"2024-09-16T04:49:46.496863+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:49:46.496863+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":3710,"log_dups_size":0,"ondisk_log_size":3710,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15070418,"num_objects":3681,"num_object_clones":0,"num_object_copies":7362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3682,"num_write_kb":14719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":385}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'3702","reported_seq":4218,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:49:46.515042+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:49:46.515042+0000","last_peered":"2024-09-16T04:49:46.515042+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:49:46.515042+0000","last_undegraded":"2024-09-16T04:49:46.515042+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":3702,"log_dups_size":0,"ondisk_log_size":3702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15138816,"num_objects":3696,"num_object_clones":0,"num_object_copies":7392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3161,"num_objects_unfound":0,"num_objects_dirty":3696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":425,"num_write_kb":1700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":749568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3161},{"shards":"2","objects":535}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'3753","reported_seq":3846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.381645+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:49:46.381645+0000","last_peered":"2024-09-16T04:49:46.381645+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:49:46.381645+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3753,"log_dups_size":0,"ondisk_log_size":3753,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15364713,"num_objects":3752,"num_object_clones":0,"num_object_copies":7504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3752,"num_write_kb":15005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":345}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'3863","reported_seq":3961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.498668+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:49:46.498668+0000","last_peered":"2024-09-16T04:49:46.498668+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:49:46.498668+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":3863,"log_dups_size":0,"ondisk_log_size":3863,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15791931,"num_objects":3858,"num_object_clones":0,"num_object_copies":7716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3860,"num_write_kb":15424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":344}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'3730","reported_seq":3853,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.087735+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:49:47.087735+0000","last_peered":"2024-09-16T04:49:47.087735+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:49:47.087735+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":3730,"log_dups_size":0,"ondisk_log_size":3730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15261696,"num_objects":3726,"num_object_clones":0,"num_object_copies":7452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":484,"num_write_kb":1936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":401},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'3920","reported_seq":4055,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.104076+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:49:47.104076+0000","last_peered":"2024-09-16T04:49:47.104076+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:49:47.104076+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":3920,"log_dups_size":0,"ondisk_log_size":3920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16007785,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":689,"num_write_kb":2756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":385},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":245711071,"num_objects":59994,"num_object_clones":0,"num_object_copies":119988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3161,"num_objects_unfound":0,"num_objects_dirty":59994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25228,"num_write_kb":100884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":761856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":487546880,"data_stored":487459905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60150,"ondisk_log_size":60150,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":379108,"kb_used_data":230804,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93992732,"statfs":{"total":96636764160,"available":96248557568,"internally_reserved":0,"allocated":236343296,"data_stored":236084534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":151842473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,2,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593731,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71552,"kb_used_data":13488,"kb_used_omap":4,"kb_used_meta":58043,"kb_avail":94300288,"statfs":{"total":96636764160,"available":96563494912,"internally_reserved":0,"allocated":13811712,"data_stored":13476412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":59436718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376056,"kb_used_data":231976,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93995784,"statfs":{"total":96636764160,"available":96251682816,"internally_reserved":0,"allocated":237543424,"data_stored":237284662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":237330432,"data_stored":237326953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13987840,"data_stored":13907823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":236228608,"data_stored":236225129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:48.932 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:49.054 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 52 20 61530 61510 4.61986 2.54688 0.0322408 0.0169006 2024-09-16T04:49:49.915 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:49.915 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:49.957 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-16T04:49:48.280443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":245711090,"num_objects":59996,"num_object_clones":0,"num_object_copies":119992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3161,"num_objects_unfound":0,"num_objects_dirty":59996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25230,"num_write_kb":100886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":761875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60153,"ondisk_log_size":60153,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":826716,"kb_used_data":476268,"kb_used_omap":12,"kb_used_meta":350387,"kb_avail":282288804,"statfs":{"total":289910292480,"available":289063735296,"internally_reserved":0,"allocated":487698432,"data_stored":486845608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":358796289},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,3,4,8,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23990272,"num_objects":5857,"num_object_clones":0,"num_object_copies":11714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3161,"num_objects_unfound":0,"num_objects_dirty":5857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5857,"num_write_kb":23428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":749568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.643278"},"pg_stats":[{"pgid":"2.f","version":"45'3738","reported_seq":3810,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.099662+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:49:47.099662+0000","last_peered":"2024-09-16T04:49:47.099662+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:49:47.099662+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3738,"log_dups_size":0,"ondisk_log_size":3738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15310848,"num_objects":3738,"num_object_clones":0,"num_object_copies":7476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3738,"num_write_kb":14952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":397},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'3755","reported_seq":3866,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:46.483882+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:49:46.483882+0000","last_peered":"2024-09-16T04:49:46.483882+0000","last_clean":"2024-09-16T04:49:46.483882+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:46.483882+0000","last_undegraded":"2024-09-16T04:49:46.483882+0000","last_fullsized":"2024-09-16T04:49:46.483882+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":3755,"log_dups_size":0,"ondisk_log_size":3755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15355904,"num_objects":3749,"num_object_clones":0,"num_object_copies":7498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":724,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'3676","reported_seq":3809,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.491100+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:49:46.491100+0000","last_peered":"2024-09-16T04:49:46.491100+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:49:46.491100+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":3676,"log_dups_size":0,"ondisk_log_size":3676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15040512,"num_objects":3672,"num_object_clones":0,"num_object_copies":7344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":415,"num_write_kb":1660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":353}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'3743","reported_seq":3808,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:46.414317+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:46.414317+0000","last_peered":"2024-09-16T04:49:46.414317+0000","last_clean":"2024-09-16T04:49:46.414317+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:46.414317+0000","last_undegraded":"2024-09-16T04:49:46.414317+0000","last_fullsized":"2024-09-16T04:49:46.414317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15331328,"num_objects":3743,"num_object_clones":0,"num_object_copies":7486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3743,"num_write_kb":14972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'3755","reported_seq":3969,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:47.073660+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:49:47.073660+0000","last_peered":"2024-09-16T04:49:47.073660+0000","last_clean":"2024-09-16T04:49:47.073660+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:47.073660+0000","last_undegraded":"2024-09-16T04:49:47.073660+0000","last_fullsized":"2024-09-16T04:49:47.073660+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":3755,"log_dups_size":0,"ondisk_log_size":3755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15257600,"num_objects":3725,"num_object_clones":0,"num_object_copies":7450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":787,"num_write_kb":3148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'3816","reported_seq":3944,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.042800+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:49:47.042800+0000","last_peered":"2024-09-16T04:49:47.042800+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:49:47.042800+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":3816,"log_dups_size":0,"ondisk_log_size":3816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15609856,"num_objects":3811,"num_object_clones":0,"num_object_copies":7622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":809,"num_write_kb":3236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":355},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'3768","reported_seq":3911,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.034518+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:49:47.034518+0000","last_peered":"2024-09-16T04:49:47.034518+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:49:47.034518+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":3768,"log_dups_size":0,"ondisk_log_size":3768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15409152,"num_objects":3762,"num_object_clones":0,"num_object_copies":7524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":423,"num_write_kb":1692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":363},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'3691","reported_seq":3902,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:47.067403+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:49:47.067403+0000","last_peered":"2024-09-16T04:49:47.067403+0000","last_clean":"2024-09-16T04:49:47.067403+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:47.067403+0000","last_undegraded":"2024-09-16T04:49:47.067403+0000","last_fullsized":"2024-09-16T04:49:47.067403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":3691,"log_dups_size":0,"ondisk_log_size":3691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15085568,"num_objects":3683,"num_object_clones":0,"num_object_copies":7366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":761,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'3683","reported_seq":3906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.475890+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:49:46.475890+0000","last_peered":"2024-09-16T04:49:46.475890+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:49:46.475890+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":3683,"log_dups_size":0,"ondisk_log_size":3683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14950400,"num_objects":3650,"num_object_clones":0,"num_object_copies":7300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":434,"num_write_kb":1736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":329}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'3847","reported_seq":3972,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.102435+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:49:47.102435+0000","last_peered":"2024-09-16T04:49:47.102435+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:49:47.102435+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":3847,"log_dups_size":0,"ondisk_log_size":3847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15724544,"num_objects":3839,"num_object_clones":0,"num_object_copies":7678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":502,"num_write_kb":2008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":374},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'3710","reported_seq":3947,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.496863+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:49:46.496863+0000","last_peered":"2024-09-16T04:49:46.496863+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:49:46.496863+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":3710,"log_dups_size":0,"ondisk_log_size":3710,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15070418,"num_objects":3681,"num_object_clones":0,"num_object_copies":7362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3682,"num_write_kb":14719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":385}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'3702","reported_seq":4218,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:49:46.515042+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:49:46.515042+0000","last_peered":"2024-09-16T04:49:46.515042+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:49:46.515042+0000","last_undegraded":"2024-09-16T04:49:46.515042+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":3702,"log_dups_size":0,"ondisk_log_size":3702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15138816,"num_objects":3696,"num_object_clones":0,"num_object_copies":7392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3161,"num_objects_unfound":0,"num_objects_dirty":3696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":425,"num_write_kb":1700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":749568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3161},{"shards":"2","objects":535}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'3753","reported_seq":3846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.381645+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:49:46.381645+0000","last_peered":"2024-09-16T04:49:46.381645+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:49:46.381645+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3753,"log_dups_size":0,"ondisk_log_size":3753,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15364713,"num_objects":3752,"num_object_clones":0,"num_object_copies":7504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3752,"num_write_kb":15005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":345}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'3863","reported_seq":3961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.498668+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:49:46.498668+0000","last_peered":"2024-09-16T04:49:46.498668+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:49:46.498668+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":3863,"log_dups_size":0,"ondisk_log_size":3863,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15791931,"num_objects":3858,"num_object_clones":0,"num_object_copies":7716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3860,"num_write_kb":15424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":344}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'3730","reported_seq":3853,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.087735+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:49:47.087735+0000","last_peered":"2024-09-16T04:49:47.087735+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:49:47.087735+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":3730,"log_dups_size":0,"ondisk_log_size":3730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15261696,"num_objects":3726,"num_object_clones":0,"num_object_copies":7452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":484,"num_write_kb":1936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":401},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'3920","reported_seq":4055,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.104076+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:49:47.104076+0000","last_peered":"2024-09-16T04:49:47.104076+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:49:47.104076+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":3920,"log_dups_size":0,"ondisk_log_size":3920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16007785,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":689,"num_write_kb":2756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":385},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":245711071,"num_objects":59994,"num_object_clones":0,"num_object_copies":119988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3161,"num_objects_unfound":0,"num_objects_dirty":59994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25228,"num_write_kb":100884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":761856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":487546880,"data_stored":487459905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60150,"ondisk_log_size":60150,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":379108,"kb_used_data":230804,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93992732,"statfs":{"total":96636764160,"available":96248557568,"internally_reserved":0,"allocated":236343296,"data_stored":236084534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":151842473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,2,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593731,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71552,"kb_used_data":13488,"kb_used_omap":4,"kb_used_meta":58043,"kb_avail":94300288,"statfs":{"total":96636764160,"available":96563494912,"internally_reserved":0,"allocated":13811712,"data_stored":13476412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":59436718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376056,"kb_used_data":231976,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93995784,"statfs":{"total":96636764160,"available":96251682816,"internally_reserved":0,"allocated":237543424,"data_stored":237284662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":237330432,"data_stored":237326953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13987840,"data_stored":13907823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":236228608,"data_stored":236225129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:49.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:49:49.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'3738", 'reported_seq': 3810, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:47.099662+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:49:47.099662+0000', 'last_peered': '2024-09-16T04:49:47.099662+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:49:47.099662+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 3738, 'log_dups_size': 0, 'ondisk_log_size': 3738, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15310848, 'num_objects': 3738, 'num_object_clones': 0, 'num_object_copies': 7476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3738, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3738, 'num_write_kb': 14952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 397}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:49.959 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:49:49.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'3676", 'reported_seq': 3809, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:46.491100+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:49:46.491100+0000', 'last_peered': '2024-09-16T04:49:46.491100+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:49:46.491100+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 3676, 'log_dups_size': 0, 'ondisk_log_size': 3676, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15040512, 'num_objects': 3672, 'num_object_clones': 0, 'num_object_copies': 7344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 415, 'num_write_kb': 1660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 353}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:49.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:49:49.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'3816", 'reported_seq': 3944, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:47.042800+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:49:47.042800+0000', 'last_peered': '2024-09-16T04:49:47.042800+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:49:47.042800+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 3816, 'log_dups_size': 0, 'ondisk_log_size': 3816, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15609856, 'num_objects': 3811, 'num_object_clones': 0, 'num_object_copies': 7622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3811, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 809, 'num_write_kb': 3236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 355}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:49.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:49:49.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'3768", 'reported_seq': 3911, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:47.034518+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:49:47.034518+0000', 'last_peered': '2024-09-16T04:49:47.034518+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:49:47.034518+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 3768, 'log_dups_size': 0, 'ondisk_log_size': 3768, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15409152, 'num_objects': 3762, 'num_object_clones': 0, 'num_object_copies': 7524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 423, 'num_write_kb': 1692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 363}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:49.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:49:49.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'3683", 'reported_seq': 3906, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:46.475890+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:49:46.475890+0000', 'last_peered': '2024-09-16T04:49:46.475890+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:49:46.475890+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 3683, 'log_dups_size': 0, 'ondisk_log_size': 3683, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14950400, 'num_objects': 3650, 'num_object_clones': 0, 'num_object_copies': 7300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 434, 'num_write_kb': 1736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 329}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:49.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:49:49.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'3847", 'reported_seq': 3972, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:47.102435+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:49:47.102435+0000', 'last_peered': '2024-09-16T04:49:47.102435+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:49:47.102435+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 3847, 'log_dups_size': 0, 'ondisk_log_size': 3847, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15724544, 'num_objects': 3839, 'num_object_clones': 0, 'num_object_copies': 7678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3839, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 502, 'num_write_kb': 2008, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 374}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:49.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:49:49.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'3710", 'reported_seq': 3947, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:46.496863+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:49:46.496863+0000', 'last_peered': '2024-09-16T04:49:46.496863+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:49:46.496863+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 3710, 'log_dups_size': 0, 'ondisk_log_size': 3710, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15070418, 'num_objects': 3681, 'num_object_clones': 0, 'num_object_copies': 7362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3681, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3682, 'num_write_kb': 14719, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 385}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:49.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:49:49.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'3702", 'reported_seq': 4218, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:49:46.515042+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:49:46.515042+0000', 'last_peered': '2024-09-16T04:49:46.515042+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:49:46.515042+0000', 'last_undegraded': '2024-09-16T04:49:46.515042+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 3702, 'log_dups_size': 0, 'ondisk_log_size': 3702, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15138816, 'num_objects': 3696, 'num_object_clones': 0, 'num_object_copies': 7392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3161, 'num_objects_unfound': 0, 'num_objects_dirty': 3696, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 425, 'num_write_kb': 1700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 183, 'num_bytes_recovered': 749568, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3161}, {'shards': '2', 'objects': 535}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:49.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:49:49.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'3753", 'reported_seq': 3846, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:46.381645+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:49:46.381645+0000', 'last_peered': '2024-09-16T04:49:46.381645+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:49:46.381645+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 3753, 'log_dups_size': 0, 'ondisk_log_size': 3753, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15364713, 'num_objects': 3752, 'num_object_clones': 0, 'num_object_copies': 7504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3752, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3752, 'num_write_kb': 15005, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 345}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:49.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:49:49.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'3863", 'reported_seq': 3961, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:46.498668+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:49:46.498668+0000', 'last_peered': '2024-09-16T04:49:46.498668+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:49:46.498668+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 3863, 'log_dups_size': 0, 'ondisk_log_size': 3863, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15791931, 'num_objects': 3858, 'num_object_clones': 0, 'num_object_copies': 7716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3858, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3860, 'num_write_kb': 15424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 344}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:49.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:49:49.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'3730", 'reported_seq': 3853, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:47.087735+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:49:47.087735+0000', 'last_peered': '2024-09-16T04:49:47.087735+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:49:47.087735+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 3730, 'log_dups_size': 0, 'ondisk_log_size': 3730, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15261696, 'num_objects': 3726, 'num_object_clones': 0, 'num_object_copies': 7452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 484, 'num_write_kb': 1936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 401}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:49.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:49:49.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'3920", 'reported_seq': 4055, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:47.104076+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:49:47.104076+0000', 'last_peered': '2024-09-16T04:49:47.104076+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:49:47.104076+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 3920, 'log_dups_size': 0, 'ondisk_log_size': 3920, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16007785, 'num_objects': 3909, 'num_object_clones': 0, 'num_object_copies': 7818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3909, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 689, 'num_write_kb': 2756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 385}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:49.963 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:49:50.054 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 53 20 62192 62172 4.58148 2.58594 0.0513529 0.0170418 2024-09-16T04:49:51.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 54 20 62815 62795 4.5417 2.43359 0.0238139 0.0171928 2024-09-16T04:49:51.114 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:51.114 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/119992 objects degraded (31.158%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":138,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":59996,"data_bytes":245711090,"bytes_used":848982016,"bytes_avail":289061310464,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":119992,"degraded_ratio":0.31157910527368493,"misplaced_objects":3161,"misplaced_total":119992,"misplaced_ratio":0.026343422894859657,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":62443,"recovering_keys_per_sec":0,"num_objects_recovered":183,"num_bytes_recovered":749568,"num_keys_recovered":0,"write_bytes_sec":1998539,"read_op_per_sec":0,"write_op_per_sec":487},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-16T04:49:42.278701+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T04:49:51.222 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:49:51.222 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:52.004 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:52.005 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:52.053 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-16T04:49:50.280929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":245711090,"num_objects":59996,"num_object_clones":0,"num_object_copies":119992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3161,"num_objects_unfound":0,"num_objects_dirty":59996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25230,"num_write_kb":100886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":761875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60153,"ondisk_log_size":60153,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":829084,"kb_used_data":486956,"kb_used_omap":12,"kb_used_meta":342067,"kb_avail":282286436,"statfs":{"total":289910292480,"available":289061310464,"internally_reserved":0,"allocated":498642944,"data_stored":497790120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":350276609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,2,5,7,10,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23990272,"num_objects":5857,"num_object_clones":0,"num_object_copies":11714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3161,"num_objects_unfound":0,"num_objects_dirty":5857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5857,"num_write_kb":23428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":749568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003899"},"pg_stats":[{"pgid":"2.f","version":"45'3738","reported_seq":3810,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.099662+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:49:47.099662+0000","last_peered":"2024-09-16T04:49:47.099662+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:49:47.099662+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3738,"log_dups_size":0,"ondisk_log_size":3738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15310848,"num_objects":3738,"num_object_clones":0,"num_object_copies":7476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3738,"num_write_kb":14952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":397},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'3755","reported_seq":3866,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:46.483882+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:49:46.483882+0000","last_peered":"2024-09-16T04:49:46.483882+0000","last_clean":"2024-09-16T04:49:46.483882+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:46.483882+0000","last_undegraded":"2024-09-16T04:49:46.483882+0000","last_fullsized":"2024-09-16T04:49:46.483882+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":3755,"log_dups_size":0,"ondisk_log_size":3755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15355904,"num_objects":3749,"num_object_clones":0,"num_object_copies":7498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":724,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'3676","reported_seq":3809,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.491100+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:49:46.491100+0000","last_peered":"2024-09-16T04:49:46.491100+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:49:46.491100+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":3676,"log_dups_size":0,"ondisk_log_size":3676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15040512,"num_objects":3672,"num_object_clones":0,"num_object_copies":7344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":415,"num_write_kb":1660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":353}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'3743","reported_seq":3808,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:46.414317+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:46.414317+0000","last_peered":"2024-09-16T04:49:46.414317+0000","last_clean":"2024-09-16T04:49:46.414317+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:46.414317+0000","last_undegraded":"2024-09-16T04:49:46.414317+0000","last_fullsized":"2024-09-16T04:49:46.414317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15331328,"num_objects":3743,"num_object_clones":0,"num_object_copies":7486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3743,"num_write_kb":14972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'3755","reported_seq":3969,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:47.073660+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:49:47.073660+0000","last_peered":"2024-09-16T04:49:47.073660+0000","last_clean":"2024-09-16T04:49:47.073660+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:47.073660+0000","last_undegraded":"2024-09-16T04:49:47.073660+0000","last_fullsized":"2024-09-16T04:49:47.073660+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":3755,"log_dups_size":0,"ondisk_log_size":3755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15257600,"num_objects":3725,"num_object_clones":0,"num_object_copies":7450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":787,"num_write_kb":3148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'3816","reported_seq":3944,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.042800+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:49:47.042800+0000","last_peered":"2024-09-16T04:49:47.042800+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:49:47.042800+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":3816,"log_dups_size":0,"ondisk_log_size":3816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15609856,"num_objects":3811,"num_object_clones":0,"num_object_copies":7622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":809,"num_write_kb":3236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":355},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'3768","reported_seq":3911,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.034518+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:49:47.034518+0000","last_peered":"2024-09-16T04:49:47.034518+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:49:47.034518+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":3768,"log_dups_size":0,"ondisk_log_size":3768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15409152,"num_objects":3762,"num_object_clones":0,"num_object_copies":7524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":423,"num_write_kb":1692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":363},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'3691","reported_seq":3902,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:47.067403+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:49:47.067403+0000","last_peered":"2024-09-16T04:49:47.067403+0000","last_clean":"2024-09-16T04:49:47.067403+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:47.067403+0000","last_undegraded":"2024-09-16T04:49:47.067403+0000","last_fullsized":"2024-09-16T04:49:47.067403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":3691,"log_dups_size":0,"ondisk_log_size":3691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15085568,"num_objects":3683,"num_object_clones":0,"num_object_copies":7366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":761,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'3683","reported_seq":3906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.475890+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:49:46.475890+0000","last_peered":"2024-09-16T04:49:46.475890+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:49:46.475890+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":3683,"log_dups_size":0,"ondisk_log_size":3683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14950400,"num_objects":3650,"num_object_clones":0,"num_object_copies":7300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":434,"num_write_kb":1736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":329}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'3847","reported_seq":3972,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.102435+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:49:47.102435+0000","last_peered":"2024-09-16T04:49:47.102435+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:49:47.102435+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":3847,"log_dups_size":0,"ondisk_log_size":3847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15724544,"num_objects":3839,"num_object_clones":0,"num_object_copies":7678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":502,"num_write_kb":2008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":374},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'3710","reported_seq":3947,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.496863+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:49:46.496863+0000","last_peered":"2024-09-16T04:49:46.496863+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:49:46.496863+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":3710,"log_dups_size":0,"ondisk_log_size":3710,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15070418,"num_objects":3681,"num_object_clones":0,"num_object_copies":7362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3682,"num_write_kb":14719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":385}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'3702","reported_seq":4218,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:49:46.515042+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:49:46.515042+0000","last_peered":"2024-09-16T04:49:46.515042+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:49:46.515042+0000","last_undegraded":"2024-09-16T04:49:46.515042+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":3702,"log_dups_size":0,"ondisk_log_size":3702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15138816,"num_objects":3696,"num_object_clones":0,"num_object_copies":7392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3161,"num_objects_unfound":0,"num_objects_dirty":3696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":425,"num_write_kb":1700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":749568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3161},{"shards":"2","objects":535}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'3753","reported_seq":3846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.381645+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:49:46.381645+0000","last_peered":"2024-09-16T04:49:46.381645+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:49:46.381645+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3753,"log_dups_size":0,"ondisk_log_size":3753,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15364713,"num_objects":3752,"num_object_clones":0,"num_object_copies":7504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3752,"num_write_kb":15005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":345}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'3863","reported_seq":3961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:46.498668+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:49:46.498668+0000","last_peered":"2024-09-16T04:49:46.498668+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:49:46.498668+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":3863,"log_dups_size":0,"ondisk_log_size":3863,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15791931,"num_objects":3858,"num_object_clones":0,"num_object_copies":7716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3860,"num_write_kb":15424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":344}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'3730","reported_seq":3853,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.087735+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:49:47.087735+0000","last_peered":"2024-09-16T04:49:47.087735+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:49:47.087735+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":3730,"log_dups_size":0,"ondisk_log_size":3730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15261696,"num_objects":3726,"num_object_clones":0,"num_object_copies":7452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":484,"num_write_kb":1936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":401},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'3920","reported_seq":4055,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:47.104076+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:49:47.104076+0000","last_peered":"2024-09-16T04:49:47.104076+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:49:47.104076+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":3920,"log_dups_size":0,"ondisk_log_size":3920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16007785,"num_objects":3909,"num_object_clones":0,"num_object_copies":7818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":689,"num_write_kb":2756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":385},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":245711071,"num_objects":59994,"num_object_clones":0,"num_object_copies":119988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3161,"num_objects_unfound":0,"num_objects_dirty":59994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25228,"num_write_kb":100884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":761856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":498651136,"data_stored":498564161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60150,"ondisk_log_size":60150,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":379108,"kb_used_data":230804,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93992732,"statfs":{"total":96636764160,"available":96248557568,"internally_reserved":0,"allocated":236343296,"data_stored":236084534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":151842473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,2,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593732,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73920,"kb_used_data":24176,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94297920,"statfs":{"total":96636764160,"available":96561070080,"internally_reserved":0,"allocated":24756224,"data_stored":24420924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50917038},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376056,"kb_used_data":231976,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93995784,"statfs":{"total":96636764160,"available":96251682816,"internally_reserved":0,"allocated":237543424,"data_stored":237284662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":237330432,"data_stored":237326953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25092096,"data_stored":25012079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":236228608,"data_stored":236225129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:52.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 55 20 63462 63442 4.50508 2.52734 0.0638293 0.0173314 2024-09-16T04:49:53.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 56 20 64167 64147 4.4738 2.75391 0.0147358 0.0174516 2024-09-16T04:49:54.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 57 19 64870 64851 4.44356 2.75 0.0165406 0.0175726 2024-09-16T04:49:55.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 58 20 65556 65536 4.41308 2.67578 0.0615028 0.0176941 2024-09-16T04:49:55.055 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:55.836 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:55.841 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:55.914 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-16T04:49:54.283304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":259326194,"num_objects":63320,"num_object_clones":0,"num_object_copies":126640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3068,"num_objects_unfound":0,"num_objects_dirty":63320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28554,"num_write_kb":114182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":1142803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63477,"ondisk_log_size":63477,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":829172,"kb_used_data":503684,"kb_used_omap":12,"kb_used_meta":325427,"kb_avail":282286348,"statfs":{"total":289910292480,"available":289061220352,"internally_reserved":0,"allocated":515772416,"data_stored":514919592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":333237249},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,2,5,5,9,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27578368,"num_objects":6733,"num_object_clones":0,"num_object_copies":13466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-204,"num_objects_unfound":0,"num_objects_dirty":6733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6733,"num_write_kb":26932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":835584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004716"},"pg_stats":[{"pgid":"2.f","version":"45'3916","reported_seq":3990,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:52.118747+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:49:52.118747+0000","last_peered":"2024-09-16T04:49:52.118747+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:49:52.118747+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3916,"log_dups_size":0,"ondisk_log_size":3916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16039936,"num_objects":3916,"num_object_clones":0,"num_object_copies":7832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3916,"num_write_kb":15664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":575},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'3947","reported_seq":4058,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:51.402724+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:49:51.402724+0000","last_peered":"2024-09-16T04:49:51.402724+0000","last_clean":"2024-09-16T04:49:51.402724+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:51.402724+0000","last_undegraded":"2024-09-16T04:49:51.402724+0000","last_fullsized":"2024-09-16T04:49:51.402724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":3947,"log_dups_size":0,"ondisk_log_size":3947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16142336,"num_objects":3941,"num_object_clones":0,"num_object_copies":7882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":916,"num_write_kb":3664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'3878","reported_seq":4011,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:51.504220+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:49:51.504220+0000","last_peered":"2024-09-16T04:49:51.504220+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:49:51.504220+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":3878,"log_dups_size":0,"ondisk_log_size":3878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15867904,"num_objects":3874,"num_object_clones":0,"num_object_copies":7748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":2468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":555}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'3941","reported_seq":4004,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:51.523839+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:51.523839+0000","last_peered":"2024-09-16T04:49:51.523839+0000","last_clean":"2024-09-16T04:49:51.523839+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:51.523839+0000","last_undegraded":"2024-09-16T04:49:51.523839+0000","last_fullsized":"2024-09-16T04:49:51.523839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3941,"log_dups_size":0,"ondisk_log_size":3941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16142336,"num_objects":3941,"num_object_clones":0,"num_object_copies":7882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3941,"num_write_kb":15764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'3985","reported_seq":4199,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:52.103749+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:49:52.103749+0000","last_peered":"2024-09-16T04:49:52.103749+0000","last_clean":"2024-09-16T04:49:52.103749+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:52.103749+0000","last_undegraded":"2024-09-16T04:49:52.103749+0000","last_fullsized":"2024-09-16T04:49:52.103749+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":3985,"log_dups_size":0,"ondisk_log_size":3985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16199680,"num_objects":3955,"num_object_clones":0,"num_object_copies":7910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1017,"num_write_kb":4068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'4020","reported_seq":4148,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:52.098079+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:49:52.098079+0000","last_peered":"2024-09-16T04:49:52.098079+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:49:52.098079+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":4020,"log_dups_size":0,"ondisk_log_size":4020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16445440,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1013,"num_write_kb":4052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":559},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'3981","reported_seq":4124,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:51.976820+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:49:51.976820+0000","last_peered":"2024-09-16T04:49:51.976820+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:49:51.976820+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":3981,"log_dups_size":0,"ondisk_log_size":3981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16281600,"num_objects":3975,"num_object_clones":0,"num_object_copies":7950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":636,"num_write_kb":2544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":576},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'3905","reported_seq":4113,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:52.066963+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:49:52.066963+0000","last_peered":"2024-09-16T04:49:52.066963+0000","last_clean":"2024-09-16T04:49:52.066963+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:52.066963+0000","last_undegraded":"2024-09-16T04:49:52.066963+0000","last_fullsized":"2024-09-16T04:49:52.066963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":3905,"log_dups_size":0,"ondisk_log_size":3905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15962112,"num_objects":3897,"num_object_clones":0,"num_object_copies":7794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":975,"num_write_kb":3900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'3917","reported_seq":4140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:51.382536+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:49:51.382536+0000","last_peered":"2024-09-16T04:49:51.382536+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:49:51.382536+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":3917,"log_dups_size":0,"ondisk_log_size":3917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15908864,"num_objects":3884,"num_object_clones":0,"num_object_copies":7768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":668,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":563}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'4049","reported_seq":4171,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:52.124712+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:49:52.124712+0000","last_peered":"2024-09-16T04:49:52.124712+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:49:52.124712+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":4049,"log_dups_size":0,"ondisk_log_size":4049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16551936,"num_objects":4041,"num_object_clones":0,"num_object_copies":8082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":576},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'3920","reported_seq":4157,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:51.524411+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:49:51.524411+0000","last_peered":"2024-09-16T04:49:51.524411+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:49:51.524411+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":3920,"log_dups_size":0,"ondisk_log_size":3920,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15930578,"num_objects":3891,"num_object_clones":0,"num_object_copies":7782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3892,"num_write_kb":15559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":595}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'3911","reported_seq":4616,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:49:51.504398+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:49:51.504398+0000","last_peered":"2024-09-16T04:49:51.504398+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:49:51.504398+0000","last_undegraded":"2024-09-16T04:49:51.504398+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":3911,"log_dups_size":0,"ondisk_log_size":3911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15994880,"num_objects":3905,"num_object_clones":0,"num_object_copies":7810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3068,"num_objects_unfound":0,"num_objects_dirty":3905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":634,"num_write_kb":2536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":1130496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3068},{"shards":"2","objects":837}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'3982","reported_seq":4075,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:51.449509+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:49:51.449509+0000","last_peered":"2024-09-16T04:49:51.449509+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:49:51.449509+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3982,"log_dups_size":0,"ondisk_log_size":3982,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16302697,"num_objects":3981,"num_object_clones":0,"num_object_copies":7962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3981,"num_write_kb":15921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'4069","reported_seq":4167,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:51.503957+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:49:51.503957+0000","last_peered":"2024-09-16T04:49:51.503957+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:49:51.503957+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":4069,"log_dups_size":0,"ondisk_log_size":4069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16635707,"num_objects":4064,"num_object_clones":0,"num_object_copies":8128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4066,"num_write_kb":16248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":550}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'3930","reported_seq":4056,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:52.089472+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:49:52.089472+0000","last_peered":"2024-09-16T04:49:52.089472+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:49:52.089472+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":3930,"log_dups_size":0,"ondisk_log_size":3930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16080896,"num_objects":3926,"num_object_clones":0,"num_object_copies":7852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":2736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":601},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'4123","reported_seq":4258,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:52.113081+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:49:52.113081+0000","last_peered":"2024-09-16T04:49:52.113081+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:49:52.113081+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":4123,"log_dups_size":0,"ondisk_log_size":4123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16839273,"num_objects":4112,"num_object_clones":0,"num_object_copies":8224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":892,"num_write_kb":3568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":588},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":259326175,"num_objects":63318,"num_object_clones":0,"num_object_copies":126636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3068,"num_objects_unfound":0,"num_objects_dirty":63318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28552,"num_write_kb":114180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":1142784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":515584000,"data_stored":515497025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63474,"ondisk_log_size":63474,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":379332,"kb_used_data":239348,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93992508,"statfs":{"total":96636764160,"available":96248328192,"internally_reserved":0,"allocated":245092352,"data_stored":244833590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":143322793},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,2,5,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593732,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73920,"kb_used_data":24176,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94297920,"statfs":{"total":96636764160,"available":96561070080,"internally_reserved":0,"allocated":24756224,"data_stored":24420924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":50917038},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375920,"kb_used_data":240160,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93995920,"statfs":{"total":96636764160,"available":96251822080,"internally_reserved":0,"allocated":245923840,"data_stored":245665078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,0,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":245612544,"data_stored":245609065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25092096,"data_stored":25012079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":244879360,"data_stored":244875881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:55.916 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:56.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 59 20 66215 66195 4.38191 2.57422 0.0542099 0.01782 2024-09-16T04:49:56.813 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:56.814 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:56.871 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-16T04:49:56.283725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":259326194,"num_objects":63320,"num_object_clones":0,"num_object_copies":126640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3068,"num_objects_unfound":0,"num_objects_dirty":63320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28554,"num_write_kb":114182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":1142803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63477,"ondisk_log_size":63477,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":847784,"kb_used_data":513972,"kb_used_omap":12,"kb_used_meta":333747,"kb_avail":282267736,"statfs":{"total":289910292480,"available":289042161664,"internally_reserved":0,"allocated":526307328,"data_stored":525454504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":341756929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,3,4,8,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27578368,"num_objects":6733,"num_object_clones":0,"num_object_copies":13466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-204,"num_objects_unfound":0,"num_objects_dirty":6733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6733,"num_write_kb":26932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":835584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004557"},"pg_stats":[{"pgid":"2.f","version":"45'3916","reported_seq":3990,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:52.118747+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:49:52.118747+0000","last_peered":"2024-09-16T04:49:52.118747+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:49:52.118747+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3916,"log_dups_size":0,"ondisk_log_size":3916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16039936,"num_objects":3916,"num_object_clones":0,"num_object_copies":7832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3916,"num_write_kb":15664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":575},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'3947","reported_seq":4058,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:51.402724+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:49:51.402724+0000","last_peered":"2024-09-16T04:49:51.402724+0000","last_clean":"2024-09-16T04:49:51.402724+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:51.402724+0000","last_undegraded":"2024-09-16T04:49:51.402724+0000","last_fullsized":"2024-09-16T04:49:51.402724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":3947,"log_dups_size":0,"ondisk_log_size":3947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16142336,"num_objects":3941,"num_object_clones":0,"num_object_copies":7882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":916,"num_write_kb":3664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'3878","reported_seq":4011,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:51.504220+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:49:51.504220+0000","last_peered":"2024-09-16T04:49:51.504220+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:49:51.504220+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":3878,"log_dups_size":0,"ondisk_log_size":3878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15867904,"num_objects":3874,"num_object_clones":0,"num_object_copies":7748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":2468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":555}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'3941","reported_seq":4004,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:51.523839+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:51.523839+0000","last_peered":"2024-09-16T04:49:51.523839+0000","last_clean":"2024-09-16T04:49:51.523839+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:51.523839+0000","last_undegraded":"2024-09-16T04:49:51.523839+0000","last_fullsized":"2024-09-16T04:49:51.523839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3941,"log_dups_size":0,"ondisk_log_size":3941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16142336,"num_objects":3941,"num_object_clones":0,"num_object_copies":7882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3941,"num_write_kb":15764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'3985","reported_seq":4199,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:52.103749+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:49:52.103749+0000","last_peered":"2024-09-16T04:49:52.103749+0000","last_clean":"2024-09-16T04:49:52.103749+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:52.103749+0000","last_undegraded":"2024-09-16T04:49:52.103749+0000","last_fullsized":"2024-09-16T04:49:52.103749+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":3985,"log_dups_size":0,"ondisk_log_size":3985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16199680,"num_objects":3955,"num_object_clones":0,"num_object_copies":7910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1017,"num_write_kb":4068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'4020","reported_seq":4148,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:52.098079+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:49:52.098079+0000","last_peered":"2024-09-16T04:49:52.098079+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:49:52.098079+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":4020,"log_dups_size":0,"ondisk_log_size":4020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16445440,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1013,"num_write_kb":4052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":559},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'3981","reported_seq":4124,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:51.976820+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:49:51.976820+0000","last_peered":"2024-09-16T04:49:51.976820+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:49:51.976820+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":3981,"log_dups_size":0,"ondisk_log_size":3981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16281600,"num_objects":3975,"num_object_clones":0,"num_object_copies":7950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":636,"num_write_kb":2544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":576},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'3905","reported_seq":4113,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:52.066963+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:49:52.066963+0000","last_peered":"2024-09-16T04:49:52.066963+0000","last_clean":"2024-09-16T04:49:52.066963+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:52.066963+0000","last_undegraded":"2024-09-16T04:49:52.066963+0000","last_fullsized":"2024-09-16T04:49:52.066963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":3905,"log_dups_size":0,"ondisk_log_size":3905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15962112,"num_objects":3897,"num_object_clones":0,"num_object_copies":7794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":975,"num_write_kb":3900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'3917","reported_seq":4140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:51.382536+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:49:51.382536+0000","last_peered":"2024-09-16T04:49:51.382536+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:49:51.382536+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":3917,"log_dups_size":0,"ondisk_log_size":3917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15908864,"num_objects":3884,"num_object_clones":0,"num_object_copies":7768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":668,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":563}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'4049","reported_seq":4171,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:52.124712+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:49:52.124712+0000","last_peered":"2024-09-16T04:49:52.124712+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:49:52.124712+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":4049,"log_dups_size":0,"ondisk_log_size":4049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16551936,"num_objects":4041,"num_object_clones":0,"num_object_copies":8082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":576},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'3920","reported_seq":4157,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:51.524411+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:49:51.524411+0000","last_peered":"2024-09-16T04:49:51.524411+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:49:51.524411+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":3920,"log_dups_size":0,"ondisk_log_size":3920,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15930578,"num_objects":3891,"num_object_clones":0,"num_object_copies":7782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3892,"num_write_kb":15559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":595}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'3911","reported_seq":4616,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:49:51.504398+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:49:51.504398+0000","last_peered":"2024-09-16T04:49:51.504398+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:49:51.504398+0000","last_undegraded":"2024-09-16T04:49:51.504398+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":3911,"log_dups_size":0,"ondisk_log_size":3911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15994880,"num_objects":3905,"num_object_clones":0,"num_object_copies":7810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3068,"num_objects_unfound":0,"num_objects_dirty":3905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":634,"num_write_kb":2536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":1130496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3068},{"shards":"2","objects":837}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'3982","reported_seq":4075,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:51.449509+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:49:51.449509+0000","last_peered":"2024-09-16T04:49:51.449509+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:49:51.449509+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":3982,"log_dups_size":0,"ondisk_log_size":3982,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16302697,"num_objects":3981,"num_object_clones":0,"num_object_copies":7962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3981,"num_write_kb":15921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'4069","reported_seq":4167,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:51.503957+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:49:51.503957+0000","last_peered":"2024-09-16T04:49:51.503957+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:49:51.503957+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":4069,"log_dups_size":0,"ondisk_log_size":4069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16635707,"num_objects":4064,"num_object_clones":0,"num_object_copies":8128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4066,"num_write_kb":16248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":550}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'3930","reported_seq":4056,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:52.089472+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:49:52.089472+0000","last_peered":"2024-09-16T04:49:52.089472+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:49:52.089472+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":3930,"log_dups_size":0,"ondisk_log_size":3930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16080896,"num_objects":3926,"num_object_clones":0,"num_object_copies":7852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":2736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":601},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'4123","reported_seq":4258,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:52.113081+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:49:52.113081+0000","last_peered":"2024-09-16T04:49:52.113081+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:49:52.113081+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":4123,"log_dups_size":0,"ondisk_log_size":4123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16839273,"num_objects":4112,"num_object_clones":0,"num_object_copies":8224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":892,"num_write_kb":3568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":588},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":259326175,"num_objects":63318,"num_object_clones":0,"num_object_copies":126636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":3068,"num_objects_unfound":0,"num_objects_dirty":63318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28552,"num_write_kb":114180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":1142784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":526106624,"data_stored":526019649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63474,"ondisk_log_size":63474,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":379332,"kb_used_data":239348,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93992508,"statfs":{"total":96636764160,"available":96248328192,"internally_reserved":0,"allocated":245092352,"data_stored":244833590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":143322793},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,2,5,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593733,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92532,"kb_used_data":34464,"kb_used_omap":4,"kb_used_meta":58043,"kb_avail":94279308,"statfs":{"total":96636764160,"available":96542011392,"internally_reserved":0,"allocated":35291136,"data_stored":34955836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":59436718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":375920,"kb_used_data":240160,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93995920,"statfs":{"total":96636764160,"available":96251822080,"internally_reserved":0,"allocated":245923840,"data_stored":245665078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,0,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":245612544,"data_stored":245609065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35614720,"data_stored":35534703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":244879360,"data_stored":244875881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:56.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:49:56.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'3916", 'reported_seq': 3990, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:52.118747+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:49:52.118747+0000', 'last_peered': '2024-09-16T04:49:52.118747+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:49:52.118747+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 3916, 'log_dups_size': 0, 'ondisk_log_size': 3916, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16039936, 'num_objects': 3916, 'num_object_clones': 0, 'num_object_copies': 7832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3916, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3916, 'num_write_kb': 15664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 575}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:56.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:49:56.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'3878", 'reported_seq': 4011, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:51.504220+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:49:51.504220+0000', 'last_peered': '2024-09-16T04:49:51.504220+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:49:51.504220+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 3878, 'log_dups_size': 0, 'ondisk_log_size': 3878, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15867904, 'num_objects': 3874, 'num_object_clones': 0, 'num_object_copies': 7748, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3874, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 617, 'num_write_kb': 2468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:56.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:49:56.872 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'4020", 'reported_seq': 4148, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:52.098079+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:49:52.098079+0000', 'last_peered': '2024-09-16T04:49:52.098079+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:49:52.098079+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 4020, 'log_dups_size': 0, 'ondisk_log_size': 4020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16445440, 'num_objects': 4015, 'num_object_clones': 0, 'num_object_copies': 8030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4015, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1013, 'num_write_kb': 4052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 559}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:56.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:49:56.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'3981", 'reported_seq': 4124, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:51.976820+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:49:51.976820+0000', 'last_peered': '2024-09-16T04:49:51.976820+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:49:51.976820+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 3981, 'log_dups_size': 0, 'ondisk_log_size': 3981, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16281600, 'num_objects': 3975, 'num_object_clones': 0, 'num_object_copies': 7950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3975, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 636, 'num_write_kb': 2544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 576}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:56.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:49:56.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'3917", 'reported_seq': 4140, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:51.382536+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:49:51.382536+0000', 'last_peered': '2024-09-16T04:49:51.382536+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:49:51.382536+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 3917, 'log_dups_size': 0, 'ondisk_log_size': 3917, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15908864, 'num_objects': 3884, 'num_object_clones': 0, 'num_object_copies': 7768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3884, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 668, 'num_write_kb': 2672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 563}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:56.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:49:56.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'4049", 'reported_seq': 4171, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:52.124712+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:49:52.124712+0000', 'last_peered': '2024-09-16T04:49:52.124712+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:49:52.124712+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 4049, 'log_dups_size': 0, 'ondisk_log_size': 4049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16551936, 'num_objects': 4041, 'num_object_clones': 0, 'num_object_copies': 8082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4041, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 704, 'num_write_kb': 2816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 576}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:56.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:49:56.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'3920", 'reported_seq': 4157, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:51.524411+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:49:51.524411+0000', 'last_peered': '2024-09-16T04:49:51.524411+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:49:51.524411+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 3920, 'log_dups_size': 0, 'ondisk_log_size': 3920, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15930578, 'num_objects': 3891, 'num_object_clones': 0, 'num_object_copies': 7782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3891, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3892, 'num_write_kb': 15559, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 595}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:56.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:49:56.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'3911", 'reported_seq': 4616, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:49:51.504398+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:49:51.504398+0000', 'last_peered': '2024-09-16T04:49:51.504398+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:49:51.504398+0000', 'last_undegraded': '2024-09-16T04:49:51.504398+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 3911, 'log_dups_size': 0, 'ondisk_log_size': 3911, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15994880, 'num_objects': 3905, 'num_object_clones': 0, 'num_object_copies': 7810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3068, 'num_objects_unfound': 0, 'num_objects_dirty': 3905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 634, 'num_write_kb': 2536, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 276, 'num_bytes_recovered': 1130496, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3068}, {'shards': '2', 'objects': 837}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:56.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:49:56.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'3982", 'reported_seq': 4075, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:51.449509+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:49:51.449509+0000', 'last_peered': '2024-09-16T04:49:51.449509+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:49:51.449509+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 3982, 'log_dups_size': 0, 'ondisk_log_size': 3982, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16302697, 'num_objects': 3981, 'num_object_clones': 0, 'num_object_copies': 7962, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3981, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3981, 'num_write_kb': 15921, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:56.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:49:56.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'4069", 'reported_seq': 4167, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:51.503957+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:49:51.503957+0000', 'last_peered': '2024-09-16T04:49:51.503957+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:49:51.503957+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 4069, 'log_dups_size': 0, 'ondisk_log_size': 4069, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16635707, 'num_objects': 4064, 'num_object_clones': 0, 'num_object_copies': 8128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4064, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4066, 'num_write_kb': 16248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 550}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:49:56.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:49:56.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'3930", 'reported_seq': 4056, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:52.089472+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:49:52.089472+0000', 'last_peered': '2024-09-16T04:49:52.089472+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:49:52.089472+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 3930, 'log_dups_size': 0, 'ondisk_log_size': 3930, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16080896, 'num_objects': 3926, 'num_object_clones': 0, 'num_object_copies': 7852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3926, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 684, 'num_write_kb': 2736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 601}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:56.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:49:56.874 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'4123", 'reported_seq': 4258, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:49:52.113081+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:49:52.113081+0000', 'last_peered': '2024-09-16T04:49:52.113081+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:49:52.113081+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 4123, 'log_dups_size': 0, 'ondisk_log_size': 4123, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16839273, 'num_objects': 4112, 'num_object_clones': 0, 'num_object_copies': 8224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4112, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 892, 'num_write_kb': 3568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 588}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:49:56.874 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:49:57.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:2024-09-16T04:49:57.054561+0000 min lat: 0.00206892 max lat: 2.69414 avg lat: 0.0179522 2024-09-16T04:49:57.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T04:49:57.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 60 20 66841 66821 4.34963 2.44531 0.00894464 0.0179522 2024-09-16T04:49:57.939 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:57.939 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/126640 objects degraded (29.522%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":145,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":63320,"data_bytes":259326194,"bytes_used":868130816,"bytes_avail":289042161664,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":126640,"degraded_ratio":0.29522267845862288,"misplaced_objects":3068,"misplaced_total":126640,"misplaced_ratio":0.024226152874289326,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":69605,"recovering_keys_per_sec":0,"num_objects_recovered":204,"num_bytes_recovered":835584,"num_keys_recovered":0,"write_bytes_sec":2297324,"read_op_per_sec":0,"write_op_per_sec":560},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-16T04:49:42.278701+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T04:49:57.997 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:49:57.997 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:49:58.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 61 20 67467 67447 4.31841 2.44531 0.00939506 0.0180823 2024-09-16T04:49:58.842 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:49:58.843 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:49:58.883 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-16T04:49:58.284164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":273215730,"num_objects":66711,"num_object_clones":0,"num_object_copies":133422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2967,"num_objects_unfound":0,"num_objects_dirty":66711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31945,"num_write_kb":127746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":1556499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66868,"ondisk_log_size":66868,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":877400,"kb_used_data":531108,"kb_used_omap":12,"kb_used_meta":346227,"kb_avail":282238120,"statfs":{"total":289910292480,"available":289011834880,"internally_reserved":0,"allocated":543854592,"data_stored":543001768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":354536449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,11,5,5,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41467904,"num_objects":10124,"num_object_clones":0,"num_object_copies":20248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-305,"num_objects_unfound":0,"num_objects_dirty":10124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10124,"num_write_kb":40496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":1249280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004568"},"pg_stats":[{"pgid":"2.f","version":"45'4095","reported_seq":4168,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:57.150621+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:49:57.150621+0000","last_peered":"2024-09-16T04:49:57.150621+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:49:57.150621+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4095,"log_dups_size":0,"ondisk_log_size":4095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773120,"num_objects":4095,"num_object_clones":0,"num_object_copies":8190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4095,"num_write_kb":16380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":754},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'4166","reported_seq":4277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:56.495669+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:49:56.495669+0000","last_peered":"2024-09-16T04:49:56.495669+0000","last_clean":"2024-09-16T04:49:56.495669+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:49:56.495669+0000","last_undegraded":"2024-09-16T04:49:56.495669+0000","last_fullsized":"2024-09-16T04:49:56.495669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":4166,"log_dups_size":0,"ondisk_log_size":4166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17039360,"num_objects":4160,"num_object_clones":0,"num_object_copies":8320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1135,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'4068","reported_seq":4201,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:56.495912+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:49:56.495912+0000","last_peered":"2024-09-16T04:49:56.495912+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:49:56.495912+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":4068,"log_dups_size":0,"ondisk_log_size":4068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16646144,"num_objects":4064,"num_object_clones":0,"num_object_copies":8128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":807,"num_write_kb":3228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":745}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'4175","reported_seq":4240,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:56.476491+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:49:56.476491+0000","last_peered":"2024-09-16T04:49:56.476491+0000","last_clean":"2024-09-16T04:49:56.476491+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:49:56.476491+0000","last_undegraded":"2024-09-16T04:49:56.476491+0000","last_fullsized":"2024-09-16T04:49:56.476491+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4175,"log_dups_size":0,"ondisk_log_size":4175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17100800,"num_objects":4175,"num_object_clones":0,"num_object_copies":8350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4175,"num_write_kb":16700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'4209","reported_seq":4423,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:57.124493+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:49:57.124493+0000","last_peered":"2024-09-16T04:49:57.124493+0000","last_clean":"2024-09-16T04:49:57.124493+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:49:57.124493+0000","last_undegraded":"2024-09-16T04:49:57.124493+0000","last_fullsized":"2024-09-16T04:49:57.124493+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":4209,"log_dups_size":0,"ondisk_log_size":4209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17117184,"num_objects":4179,"num_object_clones":0,"num_object_copies":8358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1241,"num_write_kb":4964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'4256","reported_seq":4383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:57.129439+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:49:57.129439+0000","last_peered":"2024-09-16T04:49:57.129439+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:49:57.129439+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":4256,"log_dups_size":0,"ondisk_log_size":4256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17412096,"num_objects":4251,"num_object_clones":0,"num_object_copies":8502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1249,"num_write_kb":4996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":795},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'4198","reported_seq":4340,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:57.149608+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:49:57.149608+0000","last_peered":"2024-09-16T04:49:57.149608+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:49:57.149608+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":4198,"log_dups_size":0,"ondisk_log_size":4198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17170432,"num_objects":4192,"num_object_clones":0,"num_object_copies":8384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":853,"num_write_kb":3412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":793},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'4145","reported_seq":4356,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:57.113771+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:49:57.113771+0000","last_peered":"2024-09-16T04:49:57.113771+0000","last_clean":"2024-09-16T04:49:57.113771+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:49:57.113771+0000","last_undegraded":"2024-09-16T04:49:57.113771+0000","last_fullsized":"2024-09-16T04:49:57.113771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":4145,"log_dups_size":0,"ondisk_log_size":4145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16945152,"num_objects":4137,"num_object_clones":0,"num_object_copies":8274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1215,"num_write_kb":4860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'4123","reported_seq":4346,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:56.525695+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:49:56.525695+0000","last_peered":"2024-09-16T04:49:56.525695+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:49:56.525695+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":4123,"log_dups_size":0,"ondisk_log_size":4123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16752640,"num_objects":4090,"num_object_clones":0,"num_object_copies":8180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":874,"num_write_kb":3496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":769}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'4261","reported_seq":4385,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:57.079774+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:49:57.079774+0000","last_peered":"2024-09-16T04:49:57.079774+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:49:57.079774+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":4261,"log_dups_size":0,"ondisk_log_size":4261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17420288,"num_objects":4253,"num_object_clones":0,"num_object_copies":8506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":916,"num_write_kb":3664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":788},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'4135","reported_seq":4372,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:56.467161+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:49:56.467161+0000","last_peered":"2024-09-16T04:49:56.467161+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:49:56.467161+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":4135,"log_dups_size":0,"ondisk_log_size":4135,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16811218,"num_objects":4106,"num_object_clones":0,"num_object_copies":8212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4107,"num_write_kb":16419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":810}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'4134","reported_seq":5041,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:49:56.528769+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:49:56.528769+0000","last_peered":"2024-09-16T04:49:56.528769+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:49:56.528769+0000","last_undegraded":"2024-09-16T04:49:56.528769+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16908288,"num_objects":4128,"num_object_clones":0,"num_object_copies":8256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2967,"num_objects_unfound":0,"num_objects_dirty":4128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":857,"num_write_kb":3428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":377,"num_bytes_recovered":1544192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2967},{"shards":"2","objects":1161}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'4172","reported_seq":4265,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:56.473887+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:49:56.473887+0000","last_peered":"2024-09-16T04:49:56.473887+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:49:56.473887+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4172,"log_dups_size":0,"ondisk_log_size":4172,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17080937,"num_objects":4171,"num_object_clones":0,"num_object_copies":8342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4171,"num_write_kb":16681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":764}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'4270","reported_seq":4368,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:56.499603+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:49:56.499603+0000","last_peered":"2024-09-16T04:49:56.499603+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:49:56.499603+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":4270,"log_dups_size":0,"ondisk_log_size":4270,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17459003,"num_objects":4265,"num_object_clones":0,"num_object_copies":8530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4267,"num_write_kb":17052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":751}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'4145","reported_seq":4271,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:57.079796+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:49:57.079796+0000","last_peered":"2024-09-16T04:49:57.079796+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:49:57.079796+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":4145,"log_dups_size":0,"ondisk_log_size":4145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16961536,"num_objects":4141,"num_object_clones":0,"num_object_copies":8282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":899,"num_write_kb":3596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":816},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'4313","reported_seq":4448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:49:57.059522+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:49:57.059522+0000","last_peered":"2024-09-16T04:49:57.059522+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:49:57.059522+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":4313,"log_dups_size":0,"ondisk_log_size":4313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17617513,"num_objects":4302,"num_object_clones":0,"num_object_copies":8604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1082,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":778},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":273215711,"num_objects":66709,"num_object_clones":0,"num_object_copies":133418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2967,"num_objects_unfound":0,"num_objects_dirty":66709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31943,"num_write_kb":127744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":1556480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":543760384,"data_stored":543673409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66865,"ondisk_log_size":66865,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":392124,"kb_used_data":247980,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93979716,"statfs":{"total":96636764160,"available":96235229184,"internally_reserved":0,"allocated":253931520,"data_stored":253672758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":147582633},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593733,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92532,"kb_used_data":34464,"kb_used_omap":4,"kb_used_meta":58043,"kb_avail":94279308,"statfs":{"total":96636764160,"available":96542011392,"internally_reserved":0,"allocated":35291136,"data_stored":34955836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":59436718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182424,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":392744,"kb_used_data":248664,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93979096,"statfs":{"total":96636764160,"available":96234594304,"internally_reserved":0,"allocated":254631936,"data_stored":254373174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,5,2,0,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":254488576,"data_stored":254485097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":35614720,"data_stored":35534703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":253657088,"data_stored":253653609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:49:59.055 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 62 20 68101 68081 4.2887 2.47656 0.00727357 0.0182072 2024-09-16T04:50:00.056 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 63 20 68773 68753 4.26222 2.625 0.0331393 0.0183225 2024-09-16T04:50:01.057 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 64 20 69434 69414 4.23596 2.58203 0.0437253 0.0184345 2024-09-16T04:50:01.885 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:02.057 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 65 20 70088 70068 4.21006 2.55469 0.0159989 0.0185465 2024-09-16T04:50:02.600 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:02.601 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:02.671 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-16T04:50:02.285050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":286601458,"num_objects":69979,"num_object_clones":0,"num_object_copies":139958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2849,"num_objects_unfound":0,"num_objects_dirty":69979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35213,"num_write_kb":140818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":2039827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70136,"ondisk_log_size":70136,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":927244,"kb_used_data":557340,"kb_used_omap":12,"kb_used_meta":369843,"kb_avail":282188276,"statfs":{"total":289910292480,"available":288960794624,"internally_reserved":0,"allocated":570716160,"data_stored":569863336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":378719233},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,8,6,11,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40890368,"num_objects":9983,"num_object_clones":0,"num_object_copies":19966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-312,"num_objects_unfound":0,"num_objects_dirty":9983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9983,"num_write_kb":39932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":1277952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004121"},"pg_stats":[{"pgid":"2.f","version":"45'4319","reported_seq":4393,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.166843+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:02.166843+0000","last_peered":"2024-09-16T04:50:02.166843+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:02.166843+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4319,"log_dups_size":0,"ondisk_log_size":4319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17690624,"num_objects":4319,"num_object_clones":0,"num_object_copies":8638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":17276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":978},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'4329","reported_seq":4440,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:01.589018+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:01.589018+0000","last_peered":"2024-09-16T04:50:01.589018+0000","last_clean":"2024-09-16T04:50:01.589018+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:01.589018+0000","last_undegraded":"2024-09-16T04:50:01.589018+0000","last_fullsized":"2024-09-16T04:50:01.589018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":4329,"log_dups_size":0,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17707008,"num_objects":4323,"num_object_clones":0,"num_object_copies":8646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1298,"num_write_kb":5192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'4299","reported_seq":4432,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.468272+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:01.468272+0000","last_peered":"2024-09-16T04:50:01.468272+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:01.468272+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":4299,"log_dups_size":0,"ondisk_log_size":4299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17592320,"num_objects":4295,"num_object_clones":0,"num_object_copies":8590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1038,"num_write_kb":4152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":976}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'4373","reported_seq":4438,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:01.422929+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:01.422929+0000","last_peered":"2024-09-16T04:50:01.422929+0000","last_clean":"2024-09-16T04:50:01.422929+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:01.422929+0000","last_undegraded":"2024-09-16T04:50:01.422929+0000","last_fullsized":"2024-09-16T04:50:01.422929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4373,"log_dups_size":0,"ondisk_log_size":4373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17911808,"num_objects":4373,"num_object_clones":0,"num_object_copies":8746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4373,"num_write_kb":17492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'4416","reported_seq":4630,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:02.162078+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:02.162078+0000","last_peered":"2024-09-16T04:50:02.162078+0000","last_clean":"2024-09-16T04:50:02.162078+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:02.162078+0000","last_undegraded":"2024-09-16T04:50:02.162078+0000","last_fullsized":"2024-09-16T04:50:02.162078+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17965056,"num_objects":4386,"num_object_clones":0,"num_object_copies":8772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1448,"num_write_kb":5792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'4457","reported_seq":4585,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.137840+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:02.137840+0000","last_peered":"2024-09-16T04:50:02.137840+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:02.137840+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":4457,"log_dups_size":0,"ondisk_log_size":4457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18235392,"num_objects":4452,"num_object_clones":0,"num_object_copies":8904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":996},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'4380","reported_seq":4523,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.147564+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:02.147564+0000","last_peered":"2024-09-16T04:50:02.147564+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:02.147564+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":4380,"log_dups_size":0,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17915904,"num_objects":4374,"num_object_clones":0,"num_object_copies":8748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1035,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":975},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'4348","reported_seq":4559,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:02.117503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:02.117503+0000","last_peered":"2024-09-16T04:50:02.117503+0000","last_clean":"2024-09-16T04:50:02.117503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:02.117503+0000","last_undegraded":"2024-09-16T04:50:02.117503+0000","last_fullsized":"2024-09-16T04:50:02.117503+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":4348,"log_dups_size":0,"ondisk_log_size":4348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17776640,"num_objects":4340,"num_object_clones":0,"num_object_copies":8680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1418,"num_write_kb":5672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'4346","reported_seq":4568,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.568799+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:01.568799+0000","last_peered":"2024-09-16T04:50:01.568799+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:01.568799+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":4346,"log_dups_size":0,"ondisk_log_size":4346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17666048,"num_objects":4313,"num_object_clones":0,"num_object_copies":8626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1097,"num_write_kb":4388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":992}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'4477","reported_seq":4600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.136992+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:02.136992+0000","last_peered":"2024-09-16T04:50:02.136992+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:02.136992+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":4477,"log_dups_size":0,"ondisk_log_size":4477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18305024,"num_objects":4469,"num_object_clones":0,"num_object_copies":8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1132,"num_write_kb":4528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1004},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'4358","reported_seq":4595,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.552569+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:01.552569+0000","last_peered":"2024-09-16T04:50:01.552569+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:01.552569+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17724626,"num_objects":4329,"num_object_clones":0,"num_object_copies":8658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4330,"num_write_kb":17311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1033}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'4341","reported_seq":5481,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:01.578120+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:01.578120+0000","last_peered":"2024-09-16T04:50:01.578120+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:01.578120+0000","last_undegraded":"2024-09-16T04:50:01.578120+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":4341,"log_dups_size":0,"ondisk_log_size":4341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17756160,"num_objects":4335,"num_object_clones":0,"num_object_copies":8670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2849,"num_objects_unfound":0,"num_objects_dirty":4335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1064,"num_write_kb":4256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":495,"num_bytes_recovered":2027520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2849},{"shards":"2","objects":1486}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'4360","reported_seq":4452,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.588355+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:01.588355+0000","last_peered":"2024-09-16T04:50:01.588355+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:01.588355+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4360,"log_dups_size":0,"ondisk_log_size":4360,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17850985,"num_objects":4359,"num_object_clones":0,"num_object_copies":8718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4359,"num_write_kb":17433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":952}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'4477","reported_seq":4575,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.581599+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:01.581599+0000","last_peered":"2024-09-16T04:50:01.581599+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:01.581599+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":4477,"log_dups_size":0,"ondisk_log_size":4477,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18306875,"num_objects":4472,"num_object_clones":0,"num_object_copies":8944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4474,"num_write_kb":17880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":958}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'4338","reported_seq":4464,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.109099+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:02.109099+0000","last_peered":"2024-09-16T04:50:02.109099+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:02.109099+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":4338,"log_dups_size":0,"ondisk_log_size":4338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17752064,"num_objects":4334,"num_object_clones":0,"num_object_copies":8668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1092,"num_write_kb":4368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1009},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'4515","reported_seq":4649,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.158930+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:02.158930+0000","last_peered":"2024-09-16T04:50:02.158930+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:02.158930+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":4515,"log_dups_size":0,"ondisk_log_size":4515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18444905,"num_objects":4504,"num_object_clones":0,"num_object_copies":9008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1284,"num_write_kb":5136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":980},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":286601439,"num_objects":69977,"num_object_clones":0,"num_object_copies":139954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2849,"num_objects_unfound":0,"num_objects_dirty":69977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35211,"num_write_kb":140816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":2039808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":570830848,"data_stored":570743873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70133,"ondisk_log_size":70133,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":391852,"kb_used_data":256028,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93979988,"statfs":{"total":96636764160,"available":96235507712,"internally_reserved":0,"allocated":262172672,"data_stored":261913910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":139062953},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593734,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130364,"kb_used_data":44524,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94241476,"statfs":{"total":96636764160,"available":96503271424,"internally_reserved":0,"allocated":45592576,"data_stored":45257276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":87879342},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,0,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405028,"kb_used_data":256788,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93966812,"statfs":{"total":96636764160,"available":96222015488,"internally_reserved":0,"allocated":262950912,"data_stored":262692150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":262656000,"data_stored":262652521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":46129152,"data_stored":46049135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":262045696,"data_stored":262042217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:02.673 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:03.057 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 66 20 70706 70686 4.18284 2.41406 0.0154685 0.0186668 2024-09-16T04:50:03.486 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:03.486 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:03.535 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-16T04:50:02.285050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":286601458,"num_objects":69979,"num_object_clones":0,"num_object_copies":139958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2849,"num_objects_unfound":0,"num_objects_dirty":69979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35213,"num_write_kb":140818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":2039827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70136,"ondisk_log_size":70136,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":927244,"kb_used_data":557340,"kb_used_omap":12,"kb_used_meta":369843,"kb_avail":282188276,"statfs":{"total":289910292480,"available":288960794624,"internally_reserved":0,"allocated":570716160,"data_stored":569863336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":378719233},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,8,6,11,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40890368,"num_objects":9983,"num_object_clones":0,"num_object_copies":19966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-312,"num_objects_unfound":0,"num_objects_dirty":9983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9983,"num_write_kb":39932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":1277952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004121"},"pg_stats":[{"pgid":"2.f","version":"45'4319","reported_seq":4393,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.166843+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:02.166843+0000","last_peered":"2024-09-16T04:50:02.166843+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:02.166843+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4319,"log_dups_size":0,"ondisk_log_size":4319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17690624,"num_objects":4319,"num_object_clones":0,"num_object_copies":8638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":17276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":978},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'4329","reported_seq":4440,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:01.589018+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:01.589018+0000","last_peered":"2024-09-16T04:50:01.589018+0000","last_clean":"2024-09-16T04:50:01.589018+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:01.589018+0000","last_undegraded":"2024-09-16T04:50:01.589018+0000","last_fullsized":"2024-09-16T04:50:01.589018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":4329,"log_dups_size":0,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17707008,"num_objects":4323,"num_object_clones":0,"num_object_copies":8646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1298,"num_write_kb":5192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'4299","reported_seq":4432,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.468272+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:01.468272+0000","last_peered":"2024-09-16T04:50:01.468272+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:01.468272+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":4299,"log_dups_size":0,"ondisk_log_size":4299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17592320,"num_objects":4295,"num_object_clones":0,"num_object_copies":8590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1038,"num_write_kb":4152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":976}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'4373","reported_seq":4438,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:01.422929+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:01.422929+0000","last_peered":"2024-09-16T04:50:01.422929+0000","last_clean":"2024-09-16T04:50:01.422929+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:01.422929+0000","last_undegraded":"2024-09-16T04:50:01.422929+0000","last_fullsized":"2024-09-16T04:50:01.422929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4373,"log_dups_size":0,"ondisk_log_size":4373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17911808,"num_objects":4373,"num_object_clones":0,"num_object_copies":8746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4373,"num_write_kb":17492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'4416","reported_seq":4630,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:02.162078+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:02.162078+0000","last_peered":"2024-09-16T04:50:02.162078+0000","last_clean":"2024-09-16T04:50:02.162078+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:02.162078+0000","last_undegraded":"2024-09-16T04:50:02.162078+0000","last_fullsized":"2024-09-16T04:50:02.162078+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17965056,"num_objects":4386,"num_object_clones":0,"num_object_copies":8772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1448,"num_write_kb":5792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'4457","reported_seq":4585,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.137840+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:02.137840+0000","last_peered":"2024-09-16T04:50:02.137840+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:02.137840+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":4457,"log_dups_size":0,"ondisk_log_size":4457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18235392,"num_objects":4452,"num_object_clones":0,"num_object_copies":8904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":996},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'4380","reported_seq":4523,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.147564+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:02.147564+0000","last_peered":"2024-09-16T04:50:02.147564+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:02.147564+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":4380,"log_dups_size":0,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17915904,"num_objects":4374,"num_object_clones":0,"num_object_copies":8748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1035,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":975},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'4348","reported_seq":4559,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:02.117503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:02.117503+0000","last_peered":"2024-09-16T04:50:02.117503+0000","last_clean":"2024-09-16T04:50:02.117503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:02.117503+0000","last_undegraded":"2024-09-16T04:50:02.117503+0000","last_fullsized":"2024-09-16T04:50:02.117503+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":4348,"log_dups_size":0,"ondisk_log_size":4348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17776640,"num_objects":4340,"num_object_clones":0,"num_object_copies":8680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1418,"num_write_kb":5672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'4346","reported_seq":4568,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.568799+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:01.568799+0000","last_peered":"2024-09-16T04:50:01.568799+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:01.568799+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":4346,"log_dups_size":0,"ondisk_log_size":4346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17666048,"num_objects":4313,"num_object_clones":0,"num_object_copies":8626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1097,"num_write_kb":4388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":992}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'4477","reported_seq":4600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.136992+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:02.136992+0000","last_peered":"2024-09-16T04:50:02.136992+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:02.136992+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":4477,"log_dups_size":0,"ondisk_log_size":4477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18305024,"num_objects":4469,"num_object_clones":0,"num_object_copies":8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1132,"num_write_kb":4528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1004},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'4358","reported_seq":4595,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.552569+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:01.552569+0000","last_peered":"2024-09-16T04:50:01.552569+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:01.552569+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17724626,"num_objects":4329,"num_object_clones":0,"num_object_copies":8658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4330,"num_write_kb":17311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1033}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'4341","reported_seq":5481,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:01.578120+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:01.578120+0000","last_peered":"2024-09-16T04:50:01.578120+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:01.578120+0000","last_undegraded":"2024-09-16T04:50:01.578120+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":4341,"log_dups_size":0,"ondisk_log_size":4341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17756160,"num_objects":4335,"num_object_clones":0,"num_object_copies":8670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2849,"num_objects_unfound":0,"num_objects_dirty":4335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1064,"num_write_kb":4256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":495,"num_bytes_recovered":2027520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2849},{"shards":"2","objects":1486}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'4360","reported_seq":4452,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.588355+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:01.588355+0000","last_peered":"2024-09-16T04:50:01.588355+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:01.588355+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4360,"log_dups_size":0,"ondisk_log_size":4360,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17850985,"num_objects":4359,"num_object_clones":0,"num_object_copies":8718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4359,"num_write_kb":17433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":952}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'4477","reported_seq":4575,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.581599+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:01.581599+0000","last_peered":"2024-09-16T04:50:01.581599+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:01.581599+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":4477,"log_dups_size":0,"ondisk_log_size":4477,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18306875,"num_objects":4472,"num_object_clones":0,"num_object_copies":8944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4474,"num_write_kb":17880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":958}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'4338","reported_seq":4464,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.109099+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:02.109099+0000","last_peered":"2024-09-16T04:50:02.109099+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:02.109099+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":4338,"log_dups_size":0,"ondisk_log_size":4338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17752064,"num_objects":4334,"num_object_clones":0,"num_object_copies":8668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1092,"num_write_kb":4368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1009},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'4515","reported_seq":4649,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.158930+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:02.158930+0000","last_peered":"2024-09-16T04:50:02.158930+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:02.158930+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":4515,"log_dups_size":0,"ondisk_log_size":4515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18444905,"num_objects":4504,"num_object_clones":0,"num_object_copies":9008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1284,"num_write_kb":5136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":980},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":286601439,"num_objects":69977,"num_object_clones":0,"num_object_copies":139954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2849,"num_objects_unfound":0,"num_objects_dirty":69977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35211,"num_write_kb":140816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":2039808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":570830848,"data_stored":570743873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70133,"ondisk_log_size":70133,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":391852,"kb_used_data":256028,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93979988,"statfs":{"total":96636764160,"available":96235507712,"internally_reserved":0,"allocated":262172672,"data_stored":261913910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":139062953},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593734,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130364,"kb_used_data":44524,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94241476,"statfs":{"total":96636764160,"available":96503271424,"internally_reserved":0,"allocated":45592576,"data_stored":45257276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":87879342},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,0,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405028,"kb_used_data":256788,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93966812,"statfs":{"total":96636764160,"available":96222015488,"internally_reserved":0,"allocated":262950912,"data_stored":262692150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":262656000,"data_stored":262652521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":46129152,"data_stored":46049135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":262045696,"data_stored":262042217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:03.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:50:03.537 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'4319", 'reported_seq': 4393, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:02.166843+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:50:02.166843+0000', 'last_peered': '2024-09-16T04:50:02.166843+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:50:02.166843+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 4319, 'log_dups_size': 0, 'ondisk_log_size': 4319, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17690624, 'num_objects': 4319, 'num_object_clones': 0, 'num_object_copies': 8638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4319, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4319, 'num_write_kb': 17276, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 978}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:03.537 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:50:03.537 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'4299", 'reported_seq': 4432, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:01.468272+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:50:01.468272+0000', 'last_peered': '2024-09-16T04:50:01.468272+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:50:01.468272+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 4299, 'log_dups_size': 0, 'ondisk_log_size': 4299, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17592320, 'num_objects': 4295, 'num_object_clones': 0, 'num_object_copies': 8590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4295, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1038, 'num_write_kb': 4152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 976}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:03.537 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:50:03.537 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'4457", 'reported_seq': 4585, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:02.137840+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:50:02.137840+0000', 'last_peered': '2024-09-16T04:50:02.137840+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:50:02.137840+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 4457, 'log_dups_size': 0, 'ondisk_log_size': 4457, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18235392, 'num_objects': 4452, 'num_object_clones': 0, 'num_object_copies': 8904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4452, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1450, 'num_write_kb': 5800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 996}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:03.537 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:50:03.537 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'4380", 'reported_seq': 4523, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:02.147564+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:50:02.147564+0000', 'last_peered': '2024-09-16T04:50:02.147564+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:50:02.147564+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 4380, 'log_dups_size': 0, 'ondisk_log_size': 4380, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17915904, 'num_objects': 4374, 'num_object_clones': 0, 'num_object_copies': 8748, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4374, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1035, 'num_write_kb': 4140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 975}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:03.537 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:50:03.537 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'4346", 'reported_seq': 4568, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:01.568799+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:50:01.568799+0000', 'last_peered': '2024-09-16T04:50:01.568799+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:50:01.568799+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 4346, 'log_dups_size': 0, 'ondisk_log_size': 4346, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17666048, 'num_objects': 4313, 'num_object_clones': 0, 'num_object_copies': 8626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1097, 'num_write_kb': 4388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 992}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:03.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:50:03.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'4477", 'reported_seq': 4600, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:02.136992+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:50:02.136992+0000', 'last_peered': '2024-09-16T04:50:02.136992+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:50:02.136992+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 4477, 'log_dups_size': 0, 'ondisk_log_size': 4477, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18305024, 'num_objects': 4469, 'num_object_clones': 0, 'num_object_copies': 8938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4469, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1132, 'num_write_kb': 4528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1004}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:03.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:50:03.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'4358", 'reported_seq': 4595, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:01.552569+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:50:01.552569+0000', 'last_peered': '2024-09-16T04:50:01.552569+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:50:01.552569+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 4358, 'log_dups_size': 0, 'ondisk_log_size': 4358, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17724626, 'num_objects': 4329, 'num_object_clones': 0, 'num_object_copies': 8658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4329, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4330, 'num_write_kb': 17311, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 1033}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:03.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:50:03.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'4341", 'reported_seq': 5481, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:50:01.578120+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:50:01.578120+0000', 'last_peered': '2024-09-16T04:50:01.578120+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:50:01.578120+0000', 'last_undegraded': '2024-09-16T04:50:01.578120+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 4341, 'log_dups_size': 0, 'ondisk_log_size': 4341, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17756160, 'num_objects': 4335, 'num_object_clones': 0, 'num_object_copies': 8670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2849, 'num_objects_unfound': 0, 'num_objects_dirty': 4335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1064, 'num_write_kb': 4256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 495, 'num_bytes_recovered': 2027520, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2849}, {'shards': '2', 'objects': 1486}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:03.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:50:03.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'4360", 'reported_seq': 4452, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:01.588355+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:50:01.588355+0000', 'last_peered': '2024-09-16T04:50:01.588355+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:50:01.588355+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 4360, 'log_dups_size': 0, 'ondisk_log_size': 4360, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17850985, 'num_objects': 4359, 'num_object_clones': 0, 'num_object_copies': 8718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4359, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4359, 'num_write_kb': 17433, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 952}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:03.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:50:03.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'4477", 'reported_seq': 4575, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:01.581599+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:50:01.581599+0000', 'last_peered': '2024-09-16T04:50:01.581599+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:50:01.581599+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 4477, 'log_dups_size': 0, 'ondisk_log_size': 4477, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18306875, 'num_objects': 4472, 'num_object_clones': 0, 'num_object_copies': 8944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4474, 'num_write_kb': 17880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 958}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:03.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:50:03.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'4338", 'reported_seq': 4464, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:02.109099+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:50:02.109099+0000', 'last_peered': '2024-09-16T04:50:02.109099+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:50:02.109099+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 4338, 'log_dups_size': 0, 'ondisk_log_size': 4338, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17752064, 'num_objects': 4334, 'num_object_clones': 0, 'num_object_copies': 8668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4334, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1092, 'num_write_kb': 4368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1009}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:03.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:50:03.539 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'4515", 'reported_seq': 4649, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:02.158930+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:50:02.158930+0000', 'last_peered': '2024-09-16T04:50:02.158930+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:50:02.158930+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 4515, 'log_dups_size': 0, 'ondisk_log_size': 4515, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18444905, 'num_objects': 4504, 'num_object_clones': 0, 'num_object_copies': 9008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4504, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1284, 'num_write_kb': 5136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 980}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:03.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:50:04.057 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 67 20 71331 71311 4.15685 2.44141 0.0162576 0.0187857 2024-09-16T04:50:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:04.674 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/139958 objects degraded (26.713%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":152,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":69979,"data_bytes":286601458,"bytes_used":949497856,"bytes_avail":288960794624,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":139958,"degraded_ratio":0.26713013904171251,"misplaced_objects":2849,"misplaced_total":139958,"misplaced_ratio":0.020356106832049616,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":106459,"recovering_keys_per_sec":0,"num_objects_recovered":312,"num_bytes_recovered":1277952,"num_keys_recovered":0,"write_bytes_sec":3406360,"read_op_per_sec":0,"write_op_per_sec":831},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-16T04:49:42.278701+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T04:50:04.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:50:04.735 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:05.057 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 68 20 71968 71948 4.13231 2.48828 0.021255 0.0188937 2024-09-16T04:50:05.641 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:05.642 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:05.723 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-16T04:50:04.287776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":286601458,"num_objects":69979,"num_object_clones":0,"num_object_copies":139958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2849,"num_objects_unfound":0,"num_objects_dirty":69979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35213,"num_write_kb":140818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":2039827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70136,"ondisk_log_size":70136,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":927244,"kb_used_data":557340,"kb_used_omap":12,"kb_used_meta":369843,"kb_avail":282188276,"statfs":{"total":289910292480,"available":288960794624,"internally_reserved":0,"allocated":570716160,"data_stored":569863336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":378719233},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,8,6,11,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27275264,"num_objects":6659,"num_object_clones":0,"num_object_copies":13318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":6659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6659,"num_write_kb":26636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004985"},"pg_stats":[{"pgid":"2.f","version":"45'4319","reported_seq":4393,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.166843+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:02.166843+0000","last_peered":"2024-09-16T04:50:02.166843+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:02.166843+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4319,"log_dups_size":0,"ondisk_log_size":4319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17690624,"num_objects":4319,"num_object_clones":0,"num_object_copies":8638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":17276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":978},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'4329","reported_seq":4440,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:01.589018+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:01.589018+0000","last_peered":"2024-09-16T04:50:01.589018+0000","last_clean":"2024-09-16T04:50:01.589018+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:01.589018+0000","last_undegraded":"2024-09-16T04:50:01.589018+0000","last_fullsized":"2024-09-16T04:50:01.589018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":4329,"log_dups_size":0,"ondisk_log_size":4329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17707008,"num_objects":4323,"num_object_clones":0,"num_object_copies":8646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1298,"num_write_kb":5192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'4299","reported_seq":4432,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.468272+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:01.468272+0000","last_peered":"2024-09-16T04:50:01.468272+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:01.468272+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":4299,"log_dups_size":0,"ondisk_log_size":4299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17592320,"num_objects":4295,"num_object_clones":0,"num_object_copies":8590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1038,"num_write_kb":4152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":976}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'4373","reported_seq":4438,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:01.422929+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:01.422929+0000","last_peered":"2024-09-16T04:50:01.422929+0000","last_clean":"2024-09-16T04:50:01.422929+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:01.422929+0000","last_undegraded":"2024-09-16T04:50:01.422929+0000","last_fullsized":"2024-09-16T04:50:01.422929+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4373,"log_dups_size":0,"ondisk_log_size":4373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17911808,"num_objects":4373,"num_object_clones":0,"num_object_copies":8746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4373,"num_write_kb":17492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'4416","reported_seq":4630,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:02.162078+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:02.162078+0000","last_peered":"2024-09-16T04:50:02.162078+0000","last_clean":"2024-09-16T04:50:02.162078+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:02.162078+0000","last_undegraded":"2024-09-16T04:50:02.162078+0000","last_fullsized":"2024-09-16T04:50:02.162078+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":4416,"log_dups_size":0,"ondisk_log_size":4416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17965056,"num_objects":4386,"num_object_clones":0,"num_object_copies":8772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1448,"num_write_kb":5792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'4457","reported_seq":4585,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.137840+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:02.137840+0000","last_peered":"2024-09-16T04:50:02.137840+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:02.137840+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":4457,"log_dups_size":0,"ondisk_log_size":4457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18235392,"num_objects":4452,"num_object_clones":0,"num_object_copies":8904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":996},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'4380","reported_seq":4523,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.147564+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:02.147564+0000","last_peered":"2024-09-16T04:50:02.147564+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:02.147564+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":4380,"log_dups_size":0,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17915904,"num_objects":4374,"num_object_clones":0,"num_object_copies":8748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1035,"num_write_kb":4140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":975},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'4348","reported_seq":4559,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:02.117503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:02.117503+0000","last_peered":"2024-09-16T04:50:02.117503+0000","last_clean":"2024-09-16T04:50:02.117503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:02.117503+0000","last_undegraded":"2024-09-16T04:50:02.117503+0000","last_fullsized":"2024-09-16T04:50:02.117503+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":4348,"log_dups_size":0,"ondisk_log_size":4348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17776640,"num_objects":4340,"num_object_clones":0,"num_object_copies":8680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1418,"num_write_kb":5672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'4346","reported_seq":4568,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.568799+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:01.568799+0000","last_peered":"2024-09-16T04:50:01.568799+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:01.568799+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":4346,"log_dups_size":0,"ondisk_log_size":4346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17666048,"num_objects":4313,"num_object_clones":0,"num_object_copies":8626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1097,"num_write_kb":4388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":992}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'4477","reported_seq":4600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.136992+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:02.136992+0000","last_peered":"2024-09-16T04:50:02.136992+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:02.136992+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":4477,"log_dups_size":0,"ondisk_log_size":4477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18305024,"num_objects":4469,"num_object_clones":0,"num_object_copies":8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1132,"num_write_kb":4528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1004},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'4358","reported_seq":4595,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.552569+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:01.552569+0000","last_peered":"2024-09-16T04:50:01.552569+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:01.552569+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17724626,"num_objects":4329,"num_object_clones":0,"num_object_copies":8658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4330,"num_write_kb":17311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1033}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'4341","reported_seq":5481,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:01.578120+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:01.578120+0000","last_peered":"2024-09-16T04:50:01.578120+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:01.578120+0000","last_undegraded":"2024-09-16T04:50:01.578120+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":4341,"log_dups_size":0,"ondisk_log_size":4341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17756160,"num_objects":4335,"num_object_clones":0,"num_object_copies":8670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2849,"num_objects_unfound":0,"num_objects_dirty":4335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1064,"num_write_kb":4256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":495,"num_bytes_recovered":2027520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2849},{"shards":"2","objects":1486}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'4360","reported_seq":4452,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.588355+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:01.588355+0000","last_peered":"2024-09-16T04:50:01.588355+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:01.588355+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4360,"log_dups_size":0,"ondisk_log_size":4360,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17850985,"num_objects":4359,"num_object_clones":0,"num_object_copies":8718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4359,"num_write_kb":17433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":952}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'4477","reported_seq":4575,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:01.581599+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:01.581599+0000","last_peered":"2024-09-16T04:50:01.581599+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:01.581599+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":4477,"log_dups_size":0,"ondisk_log_size":4477,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18306875,"num_objects":4472,"num_object_clones":0,"num_object_copies":8944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4474,"num_write_kb":17880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":958}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'4338","reported_seq":4464,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.109099+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:02.109099+0000","last_peered":"2024-09-16T04:50:02.109099+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:02.109099+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":4338,"log_dups_size":0,"ondisk_log_size":4338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17752064,"num_objects":4334,"num_object_clones":0,"num_object_copies":8668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1092,"num_write_kb":4368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1009},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'4515","reported_seq":4649,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:02.158930+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:02.158930+0000","last_peered":"2024-09-16T04:50:02.158930+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:02.158930+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":4515,"log_dups_size":0,"ondisk_log_size":4515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18444905,"num_objects":4504,"num_object_clones":0,"num_object_copies":9008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1284,"num_write_kb":5136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":980},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":286601439,"num_objects":69977,"num_object_clones":0,"num_object_copies":139954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2849,"num_objects_unfound":0,"num_objects_dirty":69977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35211,"num_write_kb":140816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":2039808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":570830848,"data_stored":570743873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70133,"ondisk_log_size":70133,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":391852,"kb_used_data":256028,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93979988,"statfs":{"total":96636764160,"available":96235507712,"internally_reserved":0,"allocated":262172672,"data_stored":261913910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":139062953},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593734,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130364,"kb_used_data":44524,"kb_used_omap":4,"kb_used_meta":85819,"kb_avail":94241476,"statfs":{"total":96636764160,"available":96503271424,"internally_reserved":0,"allocated":45592576,"data_stored":45257276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":87879342},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,0,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405028,"kb_used_data":256788,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93966812,"statfs":{"total":96636764160,"available":96222015488,"internally_reserved":0,"allocated":262950912,"data_stored":262692150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":262656000,"data_stored":262652521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":46129152,"data_stored":46049135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":262045696,"data_stored":262042217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:06.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 69 20 72605 72585 4.1084 2.48828 0.0198073 0.0190059 2024-09-16T04:50:07.059 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 70 20 73281 73261 4.08741 2.64062 0.0297345 0.0191057 2024-09-16T04:50:08.061 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 71 20 73935 73915 4.06575 2.55469 0.0136624 0.0192048 2024-09-16T04:50:08.726 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:09.061 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 72 20 74573 74553 4.04389 2.49219 0.0287231 0.0193122 2024-09-16T04:50:09.606 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:09.659 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-16T04:50:08.289782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":299749618,"num_objects":73189,"num_object_clones":0,"num_object_copies":146378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":73189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38423,"num_write_kb":153658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":615,"num_bytes_recovered":2490387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73346,"ondisk_log_size":73346,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":956296,"kb_used_data":582232,"kb_used_omap":12,"kb_used_meta":374003,"kb_avail":282159224,"statfs":{"total":289910292480,"available":288931045376,"internally_reserved":0,"allocated":596205568,"data_stored":595352744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":382979073},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,5,3,6,6,13,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40423424,"num_objects":9869,"num_object_clones":0,"num_object_copies":19738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-329,"num_objects_unfound":0,"num_objects_dirty":9869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9869,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":1347584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006056"},"pg_stats":[{"pgid":"2.f","version":"45'4498","reported_seq":4572,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:07.128971+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:07.128971+0000","last_peered":"2024-09-16T04:50:07.128971+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:07.128971+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4498,"log_dups_size":0,"ondisk_log_size":4498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18423808,"num_objects":4498,"num_object_clones":0,"num_object_copies":8996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4498,"num_write_kb":17992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1157},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'4548","reported_seq":4659,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:06.646520+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:06.646520+0000","last_peered":"2024-09-16T04:50:06.646520+0000","last_clean":"2024-09-16T04:50:06.646520+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:06.646520+0000","last_undegraded":"2024-09-16T04:50:06.646520+0000","last_fullsized":"2024-09-16T04:50:06.646520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":4548,"log_dups_size":0,"ondisk_log_size":4548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1517,"num_write_kb":6068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'4497","reported_seq":4630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:06.669544+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:06.669544+0000","last_peered":"2024-09-16T04:50:06.669544+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:06.669544+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":4497,"log_dups_size":0,"ondisk_log_size":4497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18403328,"num_objects":4493,"num_object_clones":0,"num_object_copies":8986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1236,"num_write_kb":4944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":1174}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'4592","reported_seq":4656,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:06.642696+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:06.642696+0000","last_peered":"2024-09-16T04:50:06.642696+0000","last_clean":"2024-09-16T04:50:06.642696+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:06.642696+0000","last_undegraded":"2024-09-16T04:50:06.642696+0000","last_fullsized":"2024-09-16T04:50:06.642696+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4592,"log_dups_size":0,"ondisk_log_size":4592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18808832,"num_objects":4592,"num_object_clones":0,"num_object_copies":9184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4592,"num_write_kb":18368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'4606","reported_seq":4820,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:07.173989+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:07.173989+0000","last_peered":"2024-09-16T04:50:07.173989+0000","last_clean":"2024-09-16T04:50:07.173989+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:07.173989+0000","last_undegraded":"2024-09-16T04:50:07.173989+0000","last_fullsized":"2024-09-16T04:50:07.173989+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":4606,"log_dups_size":0,"ondisk_log_size":4606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18743296,"num_objects":4576,"num_object_clones":0,"num_object_copies":9152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1638,"num_write_kb":6552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'4670","reported_seq":4798,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:07.063944+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:07.063944+0000","last_peered":"2024-09-16T04:50:07.063944+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:07.063944+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":4670,"log_dups_size":0,"ondisk_log_size":4670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19107840,"num_objects":4665,"num_object_clones":0,"num_object_copies":9330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1663,"num_write_kb":6652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1209},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'4592","reported_seq":4735,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:07.158655+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:07.158655+0000","last_peered":"2024-09-16T04:50:07.158655+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:07.158655+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":4592,"log_dups_size":0,"ondisk_log_size":4592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18784256,"num_objects":4586,"num_object_clones":0,"num_object_copies":9172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1247,"num_write_kb":4988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1187},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'4567","reported_seq":4778,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:07.157175+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:07.157175+0000","last_peered":"2024-09-16T04:50:07.157175+0000","last_clean":"2024-09-16T04:50:07.157175+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:07.157175+0000","last_undegraded":"2024-09-16T04:50:07.157175+0000","last_fullsized":"2024-09-16T04:50:07.157175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":4567,"log_dups_size":0,"ondisk_log_size":4567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18673664,"num_objects":4559,"num_object_clones":0,"num_object_copies":9118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1637,"num_write_kb":6548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'4569","reported_seq":4792,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:06.579531+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:06.579531+0000","last_peered":"2024-09-16T04:50:06.579531+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:06.579531+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":4569,"log_dups_size":0,"ondisk_log_size":4569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18579456,"num_objects":4536,"num_object_clones":0,"num_object_copies":9072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1320,"num_write_kb":5280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":1215}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'4654","reported_seq":4779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:07.164694+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:07.164694+0000","last_peered":"2024-09-16T04:50:07.164694+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:07.164694+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":4654,"log_dups_size":0,"ondisk_log_size":4654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19030016,"num_objects":4646,"num_object_clones":0,"num_object_copies":9292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1309,"num_write_kb":5236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1181},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'4547","reported_seq":4784,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:06.648528+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:06.648528+0000","last_peered":"2024-09-16T04:50:06.648528+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:06.648528+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":4547,"log_dups_size":0,"ondisk_log_size":4547,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18498770,"num_objects":4518,"num_object_clones":0,"num_object_copies":9036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4519,"num_write_kb":18067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1222}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'4543","reported_seq":5906,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:06.657347+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:06.657347+0000","last_peered":"2024-09-16T04:50:06.657347+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:06.657347+0000","last_undegraded":"2024-09-16T04:50:06.657347+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":4543,"log_dups_size":0,"ondisk_log_size":4543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18583552,"num_objects":4537,"num_object_clones":0,"num_object_copies":9074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":4537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1266,"num_write_kb":5064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":605,"num_bytes_recovered":2478080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2739},{"shards":"2","objects":1798}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'4549","reported_seq":4642,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:06.623199+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:06.623199+0000","last_peered":"2024-09-16T04:50:06.623199+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:06.623199+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18625129,"num_objects":4548,"num_object_clones":0,"num_object_copies":9096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4548,"num_write_kb":18189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":1141}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'4658","reported_seq":4756,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:06.646804+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:06.646804+0000","last_peered":"2024-09-16T04:50:06.646804+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:06.646804+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":4658,"log_dups_size":0,"ondisk_log_size":4658,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19048251,"num_objects":4653,"num_object_clones":0,"num_object_copies":9306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":1139}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'4529","reported_seq":4655,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:07.132843+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:07.132843+0000","last_peered":"2024-09-16T04:50:07.132843+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:07.132843+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":4529,"log_dups_size":0,"ondisk_log_size":4529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18534400,"num_objects":4525,"num_object_clones":0,"num_object_copies":9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1283,"num_write_kb":5132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1200},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'4724","reported_seq":4859,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:07.164684+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:07.164684+0000","last_peered":"2024-09-16T04:50:07.164684+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:07.164684+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":4724,"log_dups_size":0,"ondisk_log_size":4724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19300969,"num_objects":4713,"num_object_clones":0,"num_object_copies":9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1493,"num_write_kb":5972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1189},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":299749599,"num_objects":73187,"num_object_clones":0,"num_object_copies":146374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":73187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38421,"num_write_kb":153656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":614,"num_bytes_recovered":2490368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":597688320,"data_stored":597601345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73343,"ondisk_log_size":73343,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404168,"kb_used_data":264184,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93967672,"statfs":{"total":96636764160,"available":96222896128,"internally_reserved":0,"allocated":270524416,"data_stored":270265654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":143322793},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,1,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593735,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153024,"kb_used_data":54704,"kb_used_omap":4,"kb_used_meta":98299,"kb_avail":94218816,"statfs":{"total":96636764160,"available":96480067584,"internally_reserved":0,"allocated":56016896,"data_stored":55681596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":100658862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182426,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399104,"kb_used_data":263344,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93972736,"statfs":{"total":96636764160,"available":96228081664,"internally_reserved":0,"allocated":269664256,"data_stored":269405494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,2,1,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":271020032,"data_stored":271016553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":56311808,"data_stored":56231791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270356480,"data_stored":270353001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:09.661 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:10.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 73 20 75185 75165 4.02105 2.39062 0.0328198 0.0194218 2024-09-16T04:50:10.650 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:10.653 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:10.705 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-16T04:50:10.290267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":299749618,"num_objects":73189,"num_object_clones":0,"num_object_copies":146378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":73189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38423,"num_write_kb":153658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":615,"num_bytes_recovered":2490387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73346,"ondisk_log_size":73346,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":958196,"kb_used_data":592452,"kb_used_omap":12,"kb_used_meta":365683,"kb_avail":282157324,"statfs":{"total":289910292480,"available":288929099776,"internally_reserved":0,"allocated":606670848,"data_stored":605818024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":374459393},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,4,6,5,13,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26533888,"num_objects":6478,"num_object_clones":0,"num_object_copies":12956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-228,"num_objects_unfound":0,"num_objects_dirty":6478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6478,"num_write_kb":25912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":933888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006103"},"pg_stats":[{"pgid":"2.f","version":"45'4498","reported_seq":4572,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:07.128971+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:07.128971+0000","last_peered":"2024-09-16T04:50:07.128971+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:07.128971+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4498,"log_dups_size":0,"ondisk_log_size":4498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18423808,"num_objects":4498,"num_object_clones":0,"num_object_copies":8996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4498,"num_write_kb":17992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1157},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'4548","reported_seq":4659,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:06.646520+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:06.646520+0000","last_peered":"2024-09-16T04:50:06.646520+0000","last_clean":"2024-09-16T04:50:06.646520+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:06.646520+0000","last_undegraded":"2024-09-16T04:50:06.646520+0000","last_fullsized":"2024-09-16T04:50:06.646520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":4548,"log_dups_size":0,"ondisk_log_size":4548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1517,"num_write_kb":6068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'4497","reported_seq":4630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:06.669544+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:06.669544+0000","last_peered":"2024-09-16T04:50:06.669544+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:06.669544+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":4497,"log_dups_size":0,"ondisk_log_size":4497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18403328,"num_objects":4493,"num_object_clones":0,"num_object_copies":8986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1236,"num_write_kb":4944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":1174}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'4592","reported_seq":4656,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:06.642696+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:06.642696+0000","last_peered":"2024-09-16T04:50:06.642696+0000","last_clean":"2024-09-16T04:50:06.642696+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:06.642696+0000","last_undegraded":"2024-09-16T04:50:06.642696+0000","last_fullsized":"2024-09-16T04:50:06.642696+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4592,"log_dups_size":0,"ondisk_log_size":4592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18808832,"num_objects":4592,"num_object_clones":0,"num_object_copies":9184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4592,"num_write_kb":18368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'4606","reported_seq":4820,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:07.173989+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:07.173989+0000","last_peered":"2024-09-16T04:50:07.173989+0000","last_clean":"2024-09-16T04:50:07.173989+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:07.173989+0000","last_undegraded":"2024-09-16T04:50:07.173989+0000","last_fullsized":"2024-09-16T04:50:07.173989+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":4606,"log_dups_size":0,"ondisk_log_size":4606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18743296,"num_objects":4576,"num_object_clones":0,"num_object_copies":9152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1638,"num_write_kb":6552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'4670","reported_seq":4798,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:07.063944+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:07.063944+0000","last_peered":"2024-09-16T04:50:07.063944+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:07.063944+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":4670,"log_dups_size":0,"ondisk_log_size":4670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19107840,"num_objects":4665,"num_object_clones":0,"num_object_copies":9330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1663,"num_write_kb":6652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1209},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'4592","reported_seq":4735,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:07.158655+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:07.158655+0000","last_peered":"2024-09-16T04:50:07.158655+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:07.158655+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":4592,"log_dups_size":0,"ondisk_log_size":4592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18784256,"num_objects":4586,"num_object_clones":0,"num_object_copies":9172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1247,"num_write_kb":4988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1187},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'4567","reported_seq":4778,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:07.157175+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:07.157175+0000","last_peered":"2024-09-16T04:50:07.157175+0000","last_clean":"2024-09-16T04:50:07.157175+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:07.157175+0000","last_undegraded":"2024-09-16T04:50:07.157175+0000","last_fullsized":"2024-09-16T04:50:07.157175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":4567,"log_dups_size":0,"ondisk_log_size":4567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18673664,"num_objects":4559,"num_object_clones":0,"num_object_copies":9118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1637,"num_write_kb":6548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'4569","reported_seq":4792,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:06.579531+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:06.579531+0000","last_peered":"2024-09-16T04:50:06.579531+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:06.579531+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":4569,"log_dups_size":0,"ondisk_log_size":4569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18579456,"num_objects":4536,"num_object_clones":0,"num_object_copies":9072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1320,"num_write_kb":5280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":1215}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'4654","reported_seq":4779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:07.164694+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:07.164694+0000","last_peered":"2024-09-16T04:50:07.164694+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:07.164694+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":4654,"log_dups_size":0,"ondisk_log_size":4654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19030016,"num_objects":4646,"num_object_clones":0,"num_object_copies":9292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1309,"num_write_kb":5236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1181},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'4547","reported_seq":4784,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:06.648528+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:06.648528+0000","last_peered":"2024-09-16T04:50:06.648528+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:06.648528+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":4547,"log_dups_size":0,"ondisk_log_size":4547,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18498770,"num_objects":4518,"num_object_clones":0,"num_object_copies":9036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4519,"num_write_kb":18067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1222}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'4543","reported_seq":5906,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:06.657347+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:06.657347+0000","last_peered":"2024-09-16T04:50:06.657347+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:06.657347+0000","last_undegraded":"2024-09-16T04:50:06.657347+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":4543,"log_dups_size":0,"ondisk_log_size":4543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18583552,"num_objects":4537,"num_object_clones":0,"num_object_copies":9074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":4537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1266,"num_write_kb":5064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":605,"num_bytes_recovered":2478080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2739},{"shards":"2","objects":1798}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'4549","reported_seq":4642,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:06.623199+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:06.623199+0000","last_peered":"2024-09-16T04:50:06.623199+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:06.623199+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18625129,"num_objects":4548,"num_object_clones":0,"num_object_copies":9096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4548,"num_write_kb":18189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":1141}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'4658","reported_seq":4756,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:06.646804+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:06.646804+0000","last_peered":"2024-09-16T04:50:06.646804+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:06.646804+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":4658,"log_dups_size":0,"ondisk_log_size":4658,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19048251,"num_objects":4653,"num_object_clones":0,"num_object_copies":9306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":1139}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'4529","reported_seq":4655,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:07.132843+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:07.132843+0000","last_peered":"2024-09-16T04:50:07.132843+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:07.132843+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":4529,"log_dups_size":0,"ondisk_log_size":4529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18534400,"num_objects":4525,"num_object_clones":0,"num_object_copies":9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1283,"num_write_kb":5132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1200},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'4724","reported_seq":4859,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:07.164684+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:07.164684+0000","last_peered":"2024-09-16T04:50:07.164684+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:07.164684+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":4724,"log_dups_size":0,"ondisk_log_size":4724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19300969,"num_objects":4713,"num_object_clones":0,"num_object_copies":9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1493,"num_write_kb":5972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1189},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":299749599,"num_objects":73187,"num_object_clones":0,"num_object_copies":146374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2739,"num_objects_unfound":0,"num_objects_dirty":73187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38421,"num_write_kb":153656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":614,"num_bytes_recovered":2490368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":608129024,"data_stored":608042049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73343,"ondisk_log_size":73343,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404168,"kb_used_data":264184,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93967672,"statfs":{"total":96636764160,"available":96222896128,"internally_reserved":0,"allocated":270524416,"data_stored":270265654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":143322793},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,1,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593736,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":154924,"kb_used_data":64924,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94216916,"statfs":{"total":96636764160,"available":96478121984,"internally_reserved":0,"allocated":66482176,"data_stored":66146876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92139182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182426,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399104,"kb_used_data":263344,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93972736,"statfs":{"total":96636764160,"available":96228081664,"internally_reserved":0,"allocated":269664256,"data_stored":269405494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":138997418},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,2,1,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":271020032,"data_stored":271016553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66752512,"data_stored":66672495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270356480,"data_stored":270353001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:10.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:50:10.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'4498", 'reported_seq': 4572, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:07.128971+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:50:07.128971+0000', 'last_peered': '2024-09-16T04:50:07.128971+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:50:07.128971+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 4498, 'log_dups_size': 0, 'ondisk_log_size': 4498, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18423808, 'num_objects': 4498, 'num_object_clones': 0, 'num_object_copies': 8996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4498, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4498, 'num_write_kb': 17992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1157}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:10.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:50:10.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'4497", 'reported_seq': 4630, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:06.669544+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:50:06.669544+0000', 'last_peered': '2024-09-16T04:50:06.669544+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:50:06.669544+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 4497, 'log_dups_size': 0, 'ondisk_log_size': 4497, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18403328, 'num_objects': 4493, 'num_object_clones': 0, 'num_object_copies': 8986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4493, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1236, 'num_write_kb': 4944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 1174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:10.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:50:10.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'4670", 'reported_seq': 4798, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:07.063944+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:50:07.063944+0000', 'last_peered': '2024-09-16T04:50:07.063944+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:50:07.063944+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 4670, 'log_dups_size': 0, 'ondisk_log_size': 4670, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19107840, 'num_objects': 4665, 'num_object_clones': 0, 'num_object_copies': 9330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4665, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1663, 'num_write_kb': 6652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1209}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:10.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:50:10.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'4592", 'reported_seq': 4735, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:07.158655+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:50:07.158655+0000', 'last_peered': '2024-09-16T04:50:07.158655+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:50:07.158655+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 4592, 'log_dups_size': 0, 'ondisk_log_size': 4592, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18784256, 'num_objects': 4586, 'num_object_clones': 0, 'num_object_copies': 9172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4586, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1247, 'num_write_kb': 4988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1187}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:10.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:50:10.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'4569", 'reported_seq': 4792, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:06.579531+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:50:06.579531+0000', 'last_peered': '2024-09-16T04:50:06.579531+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:50:06.579531+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 4569, 'log_dups_size': 0, 'ondisk_log_size': 4569, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18579456, 'num_objects': 4536, 'num_object_clones': 0, 'num_object_copies': 9072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4536, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1320, 'num_write_kb': 5280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 1215}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:10.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:50:10.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'4654", 'reported_seq': 4779, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:07.164694+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:50:07.164694+0000', 'last_peered': '2024-09-16T04:50:07.164694+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:50:07.164694+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 4654, 'log_dups_size': 0, 'ondisk_log_size': 4654, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19030016, 'num_objects': 4646, 'num_object_clones': 0, 'num_object_copies': 9292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4646, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1309, 'num_write_kb': 5236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1181}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:10.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:50:10.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'4547", 'reported_seq': 4784, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:06.648528+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:50:06.648528+0000', 'last_peered': '2024-09-16T04:50:06.648528+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:50:06.648528+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 4547, 'log_dups_size': 0, 'ondisk_log_size': 4547, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18498770, 'num_objects': 4518, 'num_object_clones': 0, 'num_object_copies': 9036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4518, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4519, 'num_write_kb': 18067, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 1222}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:10.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:50:10.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'4543", 'reported_seq': 5906, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:50:06.657347+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:50:06.657347+0000', 'last_peered': '2024-09-16T04:50:06.657347+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:50:06.657347+0000', 'last_undegraded': '2024-09-16T04:50:06.657347+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 4543, 'log_dups_size': 0, 'ondisk_log_size': 4543, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18583552, 'num_objects': 4537, 'num_object_clones': 0, 'num_object_copies': 9074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2739, 'num_objects_unfound': 0, 'num_objects_dirty': 4537, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1266, 'num_write_kb': 5064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 605, 'num_bytes_recovered': 2478080, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2739}, {'shards': '2', 'objects': 1798}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:10.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:50:10.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'4549", 'reported_seq': 4642, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:06.623199+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:50:06.623199+0000', 'last_peered': '2024-09-16T04:50:06.623199+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:50:06.623199+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 4549, 'log_dups_size': 0, 'ondisk_log_size': 4549, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18625129, 'num_objects': 4548, 'num_object_clones': 0, 'num_object_copies': 9096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4548, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4548, 'num_write_kb': 18189, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 1141}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:10.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:50:10.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'4658", 'reported_seq': 4756, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:06.646804+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:50:06.646804+0000', 'last_peered': '2024-09-16T04:50:06.646804+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:50:06.646804+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 4658, 'log_dups_size': 0, 'ondisk_log_size': 4658, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19048251, 'num_objects': 4653, 'num_object_clones': 0, 'num_object_copies': 9306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4653, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4655, 'num_write_kb': 18604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 1139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:10.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:50:10.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'4529", 'reported_seq': 4655, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:07.132843+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:50:07.132843+0000', 'last_peered': '2024-09-16T04:50:07.132843+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:50:07.132843+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 4529, 'log_dups_size': 0, 'ondisk_log_size': 4529, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18534400, 'num_objects': 4525, 'num_object_clones': 0, 'num_object_copies': 9050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4525, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1283, 'num_write_kb': 5132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1200}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:10.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:50:10.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'4724", 'reported_seq': 4859, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:07.164684+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:50:07.164684+0000', 'last_peered': '2024-09-16T04:50:07.164684+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:50:07.164684+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 4724, 'log_dups_size': 0, 'ondisk_log_size': 4724, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19300969, 'num_objects': 4713, 'num_object_clones': 0, 'num_object_copies': 9426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4713, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1493, 'num_write_kb': 5972, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1189}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:10.709 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:50:11.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 74 20 75799 75779 3.99912 2.39844 0.0290166 0.0195272 2024-09-16T04:50:11.813 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:11.813 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/146378 objects degraded (25.541%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":159,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":73189,"data_bytes":299749618,"bytes_used":981192704,"bytes_avail":288929099776,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":146378,"degraded_ratio":0.2554140649551162,"misplaced_objects":2739,"misplaced_total":146378,"misplaced_ratio":0.018711828280206042,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":77784,"recovering_keys_per_sec":0,"num_objects_recovered":228,"num_bytes_recovered":933888,"num_keys_recovered":0,"write_bytes_sec":2210033,"read_op_per_sec":0,"write_op_per_sec":539},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-16T04:49:42.278701+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T04:50:11.892 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:50:11.893 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:12.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 75 20 76420 76400 3.97814 2.42578 0.00648732 0.0196273 2024-09-16T04:50:12.600 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:12.670 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-16T04:50:12.290682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":312729842,"num_objects":76358,"num_object_clones":0,"num_object_copies":152716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2630,"num_objects_unfound":0,"num_objects_dirty":76358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41592,"num_write_kb":166334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":724,"num_bytes_recovered":2936851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76515,"ondisk_log_size":76515,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":992108,"kb_used_data":609724,"kb_used_omap":12,"kb_used_meta":382323,"kb_avail":282123412,"statfs":{"total":289910292480,"available":288894373888,"internally_reserved":0,"allocated":624357376,"data_stored":623504552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":391498753},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,7,6,9,10,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39514112,"num_objects":9647,"num_object_clones":0,"num_object_copies":19294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":9647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9647,"num_write_kb":38588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006054"},"pg_stats":[{"pgid":"2.f","version":"45'4731","reported_seq":4804,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.195941+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:12.195941+0000","last_peered":"2024-09-16T04:50:12.195941+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:12.195941+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4731,"log_dups_size":0,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19378176,"num_objects":4731,"num_object_clones":0,"num_object_copies":9462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4731,"num_write_kb":18924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1390},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'4760","reported_seq":4871,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:11.646762+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:11.646762+0000","last_peered":"2024-09-16T04:50:11.646762+0000","last_clean":"2024-09-16T04:50:11.646762+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:11.646762+0000","last_undegraded":"2024-09-16T04:50:11.646762+0000","last_fullsized":"2024-09-16T04:50:11.646762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":4760,"log_dups_size":0,"ondisk_log_size":4760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19472384,"num_objects":4754,"num_object_clones":0,"num_object_copies":9508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1729,"num_write_kb":6916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'4710","reported_seq":4843,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.680707+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:11.680707+0000","last_peered":"2024-09-16T04:50:11.680707+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:11.680707+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":4710,"log_dups_size":0,"ondisk_log_size":4710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19275776,"num_objects":4706,"num_object_clones":0,"num_object_copies":9412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1449,"num_write_kb":5796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":1387}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'4767","reported_seq":4830,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:11.663266+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:11.663266+0000","last_peered":"2024-09-16T04:50:11.663266+0000","last_clean":"2024-09-16T04:50:11.663266+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:11.663266+0000","last_undegraded":"2024-09-16T04:50:11.663266+0000","last_fullsized":"2024-09-16T04:50:11.663266+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4767,"log_dups_size":0,"ondisk_log_size":4767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4767,"num_write_kb":19068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'4810","reported_seq":5024,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:12.166141+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:12.166141+0000","last_peered":"2024-09-16T04:50:12.166141+0000","last_clean":"2024-09-16T04:50:12.166141+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:12.166141+0000","last_undegraded":"2024-09-16T04:50:12.166141+0000","last_fullsized":"2024-09-16T04:50:12.166141+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":4810,"log_dups_size":0,"ondisk_log_size":4810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19578880,"num_objects":4780,"num_object_clones":0,"num_object_copies":9560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1842,"num_write_kb":7368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'4845","reported_seq":4971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.148682+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:12.148682+0000","last_peered":"2024-09-16T04:50:12.148682+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:12.148682+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":4845,"log_dups_size":0,"ondisk_log_size":4845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19824640,"num_objects":4840,"num_object_clones":0,"num_object_copies":9680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1838,"num_write_kb":7352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1384},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'4776","reported_seq":4919,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.199464+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:12.199464+0000","last_peered":"2024-09-16T04:50:12.199464+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:12.199464+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19537920,"num_objects":4770,"num_object_clones":0,"num_object_copies":9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1431,"num_write_kb":5724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1371},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'4756","reported_seq":4965,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:12.129106+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:12.129106+0000","last_peered":"2024-09-16T04:50:12.129106+0000","last_clean":"2024-09-16T04:50:12.129106+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:12.129106+0000","last_undegraded":"2024-09-16T04:50:12.129106+0000","last_fullsized":"2024-09-16T04:50:12.129106+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":4756,"log_dups_size":0,"ondisk_log_size":4756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19447808,"num_objects":4748,"num_object_clones":0,"num_object_copies":9496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1826,"num_write_kb":7304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'4755","reported_seq":4978,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.548416+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:11.548416+0000","last_peered":"2024-09-16T04:50:11.548416+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:11.548416+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":4755,"log_dups_size":0,"ondisk_log_size":4755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19341312,"num_objects":4722,"num_object_clones":0,"num_object_copies":9444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1506,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":1401}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'4861","reported_seq":4986,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.090057+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:12.090057+0000","last_peered":"2024-09-16T04:50:12.090057+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:12.090057+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":4861,"log_dups_size":0,"ondisk_log_size":4861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19877888,"num_objects":4853,"num_object_clones":0,"num_object_copies":9706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1516,"num_write_kb":6064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1388},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'4729","reported_seq":4966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.664464+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:11.664464+0000","last_peered":"2024-09-16T04:50:11.664464+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:11.664464+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":4729,"log_dups_size":0,"ondisk_log_size":4729,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19244242,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4701,"num_write_kb":18795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1404}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'4740","reported_seq":6319,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:11.668156+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:11.668156+0000","last_peered":"2024-09-16T04:50:11.668156+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:11.668156+0000","last_undegraded":"2024-09-16T04:50:11.668156+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":4740,"log_dups_size":0,"ondisk_log_size":4740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19390464,"num_objects":4734,"num_object_clones":0,"num_object_copies":9468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2630,"num_objects_unfound":0,"num_objects_dirty":4734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1463,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":714,"num_bytes_recovered":2924544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2630},{"shards":"2","objects":2104}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'4775","reported_seq":4868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.670152+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:11.670152+0000","last_peered":"2024-09-16T04:50:11.670152+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:11.670152+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550825,"num_objects":4774,"num_object_clones":0,"num_object_copies":9548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4774,"num_write_kb":19093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":1367}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'4849","reported_seq":4947,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.642254+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:11.642254+0000","last_peered":"2024-09-16T04:50:11.642254+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:11.642254+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":4849,"log_dups_size":0,"ondisk_log_size":4849,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19830587,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4846,"num_write_kb":19368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":1330}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'4726","reported_seq":4852,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.151506+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:12.151506+0000","last_peered":"2024-09-16T04:50:12.151506+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:12.151506+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":4726,"log_dups_size":0,"ondisk_log_size":4726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19341312,"num_objects":4722,"num_object_clones":0,"num_object_copies":9444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":5920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1397},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'4922","reported_seq":5057,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.189274+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:12.189274+0000","last_peered":"2024-09-16T04:50:12.189274+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:12.189274+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":4922,"log_dups_size":0,"ondisk_log_size":4922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20111977,"num_objects":4911,"num_object_clones":0,"num_object_copies":9822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1691,"num_write_kb":6764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1387},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":312729823,"num_objects":76356,"num_object_clones":0,"num_object_copies":152712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2630,"num_objects_unfound":0,"num_objects_dirty":76356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41590,"num_write_kb":166332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":723,"num_bytes_recovered":2936832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":624263168,"data_stored":624176193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76512,"ondisk_log_size":76512,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":420444,"kb_used_data":272140,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93951396,"statfs":{"total":96636764160,"available":96206229504,"internally_reserved":0,"allocated":278671360,"data_stored":278412598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":151842473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,3,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593736,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":154924,"kb_used_data":64924,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94216916,"statfs":{"total":96636764160,"available":96478121984,"internally_reserved":0,"allocated":66482176,"data_stored":66146876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92139182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416740,"kb_used_data":272660,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93955100,"statfs":{"total":96636764160,"available":96210022400,"internally_reserved":0,"allocated":279203840,"data_stored":278945078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,0,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":279023616,"data_stored":279020137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66752512,"data_stored":66672495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278487040,"data_stored":278483561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:13.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 76 20 77028 77008 3.95705 2.375 0.0623054 0.0197344 2024-09-16T04:50:14.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 77 20 77662 77642 3.93782 2.47656 0.0564537 0.0198325 2024-09-16T04:50:15.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 78 20 78311 78291 3.91984 2.53516 0.0116012 0.0199206 2024-09-16T04:50:15.673 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:16.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 79 20 78931 78911 3.90088 2.42188 0.0146389 0.0200159 2024-09-16T04:50:16.658 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:16.658 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:16.704 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-16T04:50:16.291563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":312729842,"num_objects":76358,"num_object_clones":0,"num_object_copies":152716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2630,"num_objects_unfound":0,"num_objects_dirty":76358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41592,"num_write_kb":166334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":724,"num_bytes_recovered":2936851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76515,"ondisk_log_size":76515,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":993624,"kb_used_data":619556,"kb_used_omap":12,"kb_used_meta":374003,"kb_avail":282121896,"statfs":{"total":289910292480,"available":288892821504,"internally_reserved":0,"allocated":634425344,"data_stored":633572520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":382979073},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,6,4,9,6,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26128384,"num_objects":6379,"num_object_clones":0,"num_object_copies":12758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":6379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6379,"num_write_kb":25516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003787"},"pg_stats":[{"pgid":"2.f","version":"45'4731","reported_seq":4804,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.195941+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:12.195941+0000","last_peered":"2024-09-16T04:50:12.195941+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:12.195941+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4731,"log_dups_size":0,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19378176,"num_objects":4731,"num_object_clones":0,"num_object_copies":9462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4731,"num_write_kb":18924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1390},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'4760","reported_seq":4871,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:11.646762+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:11.646762+0000","last_peered":"2024-09-16T04:50:11.646762+0000","last_clean":"2024-09-16T04:50:11.646762+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:11.646762+0000","last_undegraded":"2024-09-16T04:50:11.646762+0000","last_fullsized":"2024-09-16T04:50:11.646762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":4760,"log_dups_size":0,"ondisk_log_size":4760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19472384,"num_objects":4754,"num_object_clones":0,"num_object_copies":9508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1729,"num_write_kb":6916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'4710","reported_seq":4843,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.680707+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:11.680707+0000","last_peered":"2024-09-16T04:50:11.680707+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:11.680707+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":4710,"log_dups_size":0,"ondisk_log_size":4710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19275776,"num_objects":4706,"num_object_clones":0,"num_object_copies":9412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1449,"num_write_kb":5796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":1387}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'4767","reported_seq":4830,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:11.663266+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:11.663266+0000","last_peered":"2024-09-16T04:50:11.663266+0000","last_clean":"2024-09-16T04:50:11.663266+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:11.663266+0000","last_undegraded":"2024-09-16T04:50:11.663266+0000","last_fullsized":"2024-09-16T04:50:11.663266+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4767,"log_dups_size":0,"ondisk_log_size":4767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4767,"num_write_kb":19068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'4810","reported_seq":5024,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:12.166141+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:12.166141+0000","last_peered":"2024-09-16T04:50:12.166141+0000","last_clean":"2024-09-16T04:50:12.166141+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:12.166141+0000","last_undegraded":"2024-09-16T04:50:12.166141+0000","last_fullsized":"2024-09-16T04:50:12.166141+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":4810,"log_dups_size":0,"ondisk_log_size":4810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19578880,"num_objects":4780,"num_object_clones":0,"num_object_copies":9560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1842,"num_write_kb":7368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'4845","reported_seq":4971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.148682+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:12.148682+0000","last_peered":"2024-09-16T04:50:12.148682+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:12.148682+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":4845,"log_dups_size":0,"ondisk_log_size":4845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19824640,"num_objects":4840,"num_object_clones":0,"num_object_copies":9680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1838,"num_write_kb":7352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1384},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'4776","reported_seq":4919,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.199464+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:12.199464+0000","last_peered":"2024-09-16T04:50:12.199464+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:12.199464+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19537920,"num_objects":4770,"num_object_clones":0,"num_object_copies":9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1431,"num_write_kb":5724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1371},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'4756","reported_seq":4965,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:12.129106+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:12.129106+0000","last_peered":"2024-09-16T04:50:12.129106+0000","last_clean":"2024-09-16T04:50:12.129106+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:12.129106+0000","last_undegraded":"2024-09-16T04:50:12.129106+0000","last_fullsized":"2024-09-16T04:50:12.129106+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":4756,"log_dups_size":0,"ondisk_log_size":4756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19447808,"num_objects":4748,"num_object_clones":0,"num_object_copies":9496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1826,"num_write_kb":7304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'4755","reported_seq":4978,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.548416+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:11.548416+0000","last_peered":"2024-09-16T04:50:11.548416+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:11.548416+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":4755,"log_dups_size":0,"ondisk_log_size":4755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19341312,"num_objects":4722,"num_object_clones":0,"num_object_copies":9444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1506,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":1401}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'4861","reported_seq":4986,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.090057+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:12.090057+0000","last_peered":"2024-09-16T04:50:12.090057+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:12.090057+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":4861,"log_dups_size":0,"ondisk_log_size":4861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19877888,"num_objects":4853,"num_object_clones":0,"num_object_copies":9706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1516,"num_write_kb":6064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1388},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'4729","reported_seq":4966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.664464+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:11.664464+0000","last_peered":"2024-09-16T04:50:11.664464+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:11.664464+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":4729,"log_dups_size":0,"ondisk_log_size":4729,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19244242,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4701,"num_write_kb":18795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1404}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'4740","reported_seq":6319,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:11.668156+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:11.668156+0000","last_peered":"2024-09-16T04:50:11.668156+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:11.668156+0000","last_undegraded":"2024-09-16T04:50:11.668156+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":4740,"log_dups_size":0,"ondisk_log_size":4740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19390464,"num_objects":4734,"num_object_clones":0,"num_object_copies":9468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2630,"num_objects_unfound":0,"num_objects_dirty":4734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1463,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":714,"num_bytes_recovered":2924544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2630},{"shards":"2","objects":2104}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'4775","reported_seq":4868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.670152+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:11.670152+0000","last_peered":"2024-09-16T04:50:11.670152+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:11.670152+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550825,"num_objects":4774,"num_object_clones":0,"num_object_copies":9548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4774,"num_write_kb":19093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":1367}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'4849","reported_seq":4947,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.642254+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:11.642254+0000","last_peered":"2024-09-16T04:50:11.642254+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:11.642254+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":4849,"log_dups_size":0,"ondisk_log_size":4849,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19830587,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4846,"num_write_kb":19368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":1330}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'4726","reported_seq":4852,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.151506+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:12.151506+0000","last_peered":"2024-09-16T04:50:12.151506+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:12.151506+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":4726,"log_dups_size":0,"ondisk_log_size":4726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19341312,"num_objects":4722,"num_object_clones":0,"num_object_copies":9444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":5920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1397},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'4922","reported_seq":5057,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.189274+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:12.189274+0000","last_peered":"2024-09-16T04:50:12.189274+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:12.189274+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":4922,"log_dups_size":0,"ondisk_log_size":4922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20111977,"num_objects":4911,"num_object_clones":0,"num_object_copies":9822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1691,"num_write_kb":6764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1387},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":312729823,"num_objects":76356,"num_object_clones":0,"num_object_copies":152712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2630,"num_objects_unfound":0,"num_objects_dirty":76356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41590,"num_write_kb":166332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":723,"num_bytes_recovered":2936832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":634298368,"data_stored":634211393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76512,"ondisk_log_size":76512,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":420444,"kb_used_data":272140,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93951396,"statfs":{"total":96636764160,"available":96206229504,"internally_reserved":0,"allocated":278671360,"data_stored":278412598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":151842473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,3,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593737,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":156440,"kb_used_data":74756,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94215400,"statfs":{"total":96636764160,"available":96476569600,"internally_reserved":0,"allocated":76550144,"data_stored":76214844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":83619502},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416740,"kb_used_data":272660,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93955100,"statfs":{"total":96636764160,"available":96210022400,"internally_reserved":0,"allocated":279203840,"data_stored":278945078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,0,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":279023616,"data_stored":279020137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":76787712,"data_stored":76707695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278487040,"data_stored":278483561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:16.706 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:17.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:2024-09-16T04:50:17.064749+0000 min lat: 0.00206892 max lat: 2.69414 avg lat: 0.0201216 2024-09-16T04:50:17.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T04:50:17.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 80 20 79520 79500 3.88088 2.30078 0.0335293 0.0201216 2024-09-16T04:50:17.460 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:17.462 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:17.533 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-16T04:50:16.291563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":312729842,"num_objects":76358,"num_object_clones":0,"num_object_copies":152716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2630,"num_objects_unfound":0,"num_objects_dirty":76358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41592,"num_write_kb":166334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":724,"num_bytes_recovered":2936851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76515,"ondisk_log_size":76515,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":993624,"kb_used_data":619556,"kb_used_omap":12,"kb_used_meta":374003,"kb_avail":282121896,"statfs":{"total":289910292480,"available":288892821504,"internally_reserved":0,"allocated":634425344,"data_stored":633572520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":382979073},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,6,4,9,6,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26128384,"num_objects":6379,"num_object_clones":0,"num_object_copies":12758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":6379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6379,"num_write_kb":25516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003787"},"pg_stats":[{"pgid":"2.f","version":"45'4731","reported_seq":4804,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.195941+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:12.195941+0000","last_peered":"2024-09-16T04:50:12.195941+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:12.195941+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4731,"log_dups_size":0,"ondisk_log_size":4731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19378176,"num_objects":4731,"num_object_clones":0,"num_object_copies":9462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4731,"num_write_kb":18924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1390},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'4760","reported_seq":4871,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:11.646762+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:11.646762+0000","last_peered":"2024-09-16T04:50:11.646762+0000","last_clean":"2024-09-16T04:50:11.646762+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:11.646762+0000","last_undegraded":"2024-09-16T04:50:11.646762+0000","last_fullsized":"2024-09-16T04:50:11.646762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":4760,"log_dups_size":0,"ondisk_log_size":4760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19472384,"num_objects":4754,"num_object_clones":0,"num_object_copies":9508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1729,"num_write_kb":6916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'4710","reported_seq":4843,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.680707+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:11.680707+0000","last_peered":"2024-09-16T04:50:11.680707+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:11.680707+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":4710,"log_dups_size":0,"ondisk_log_size":4710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19275776,"num_objects":4706,"num_object_clones":0,"num_object_copies":9412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1449,"num_write_kb":5796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":1387}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'4767","reported_seq":4830,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:11.663266+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:11.663266+0000","last_peered":"2024-09-16T04:50:11.663266+0000","last_clean":"2024-09-16T04:50:11.663266+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:11.663266+0000","last_undegraded":"2024-09-16T04:50:11.663266+0000","last_fullsized":"2024-09-16T04:50:11.663266+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4767,"log_dups_size":0,"ondisk_log_size":4767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19525632,"num_objects":4767,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4767,"num_write_kb":19068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'4810","reported_seq":5024,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:12.166141+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:12.166141+0000","last_peered":"2024-09-16T04:50:12.166141+0000","last_clean":"2024-09-16T04:50:12.166141+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:12.166141+0000","last_undegraded":"2024-09-16T04:50:12.166141+0000","last_fullsized":"2024-09-16T04:50:12.166141+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":4810,"log_dups_size":0,"ondisk_log_size":4810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19578880,"num_objects":4780,"num_object_clones":0,"num_object_copies":9560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1842,"num_write_kb":7368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'4845","reported_seq":4971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.148682+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:12.148682+0000","last_peered":"2024-09-16T04:50:12.148682+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:12.148682+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":4845,"log_dups_size":0,"ondisk_log_size":4845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19824640,"num_objects":4840,"num_object_clones":0,"num_object_copies":9680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1838,"num_write_kb":7352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1384},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'4776","reported_seq":4919,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.199464+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:12.199464+0000","last_peered":"2024-09-16T04:50:12.199464+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:12.199464+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19537920,"num_objects":4770,"num_object_clones":0,"num_object_copies":9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1431,"num_write_kb":5724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1371},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'4756","reported_seq":4965,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:12.129106+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:12.129106+0000","last_peered":"2024-09-16T04:50:12.129106+0000","last_clean":"2024-09-16T04:50:12.129106+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:12.129106+0000","last_undegraded":"2024-09-16T04:50:12.129106+0000","last_fullsized":"2024-09-16T04:50:12.129106+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":4756,"log_dups_size":0,"ondisk_log_size":4756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19447808,"num_objects":4748,"num_object_clones":0,"num_object_copies":9496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1826,"num_write_kb":7304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'4755","reported_seq":4978,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.548416+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:11.548416+0000","last_peered":"2024-09-16T04:50:11.548416+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:11.548416+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":4755,"log_dups_size":0,"ondisk_log_size":4755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19341312,"num_objects":4722,"num_object_clones":0,"num_object_copies":9444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1506,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":1401}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'4861","reported_seq":4986,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.090057+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:12.090057+0000","last_peered":"2024-09-16T04:50:12.090057+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:12.090057+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":4861,"log_dups_size":0,"ondisk_log_size":4861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19877888,"num_objects":4853,"num_object_clones":0,"num_object_copies":9706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1516,"num_write_kb":6064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1388},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'4729","reported_seq":4966,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.664464+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:11.664464+0000","last_peered":"2024-09-16T04:50:11.664464+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:11.664464+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":4729,"log_dups_size":0,"ondisk_log_size":4729,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19244242,"num_objects":4700,"num_object_clones":0,"num_object_copies":9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4701,"num_write_kb":18795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1404}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'4740","reported_seq":6319,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:11.668156+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:11.668156+0000","last_peered":"2024-09-16T04:50:11.668156+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:11.668156+0000","last_undegraded":"2024-09-16T04:50:11.668156+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":4740,"log_dups_size":0,"ondisk_log_size":4740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19390464,"num_objects":4734,"num_object_clones":0,"num_object_copies":9468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2630,"num_objects_unfound":0,"num_objects_dirty":4734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1463,"num_write_kb":5852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":714,"num_bytes_recovered":2924544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2630},{"shards":"2","objects":2104}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'4775","reported_seq":4868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.670152+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:11.670152+0000","last_peered":"2024-09-16T04:50:11.670152+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:11.670152+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550825,"num_objects":4774,"num_object_clones":0,"num_object_copies":9548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4774,"num_write_kb":19093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":1367}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'4849","reported_seq":4947,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:11.642254+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:11.642254+0000","last_peered":"2024-09-16T04:50:11.642254+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:11.642254+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":4849,"log_dups_size":0,"ondisk_log_size":4849,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19830587,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4846,"num_write_kb":19368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":1330}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'4726","reported_seq":4852,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.151506+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:12.151506+0000","last_peered":"2024-09-16T04:50:12.151506+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:12.151506+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":4726,"log_dups_size":0,"ondisk_log_size":4726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19341312,"num_objects":4722,"num_object_clones":0,"num_object_copies":9444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1480,"num_write_kb":5920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1397},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'4922","reported_seq":5057,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:12.189274+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:12.189274+0000","last_peered":"2024-09-16T04:50:12.189274+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:12.189274+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":4922,"log_dups_size":0,"ondisk_log_size":4922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20111977,"num_objects":4911,"num_object_clones":0,"num_object_copies":9822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1691,"num_write_kb":6764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1387},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":312729823,"num_objects":76356,"num_object_clones":0,"num_object_copies":152712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2630,"num_objects_unfound":0,"num_objects_dirty":76356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41590,"num_write_kb":166332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":723,"num_bytes_recovered":2936832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":634298368,"data_stored":634211393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76512,"ondisk_log_size":76512,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":420444,"kb_used_data":272140,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93951396,"statfs":{"total":96636764160,"available":96206229504,"internally_reserved":0,"allocated":278671360,"data_stored":278412598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":151842473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,3,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593737,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":156440,"kb_used_data":74756,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94215400,"statfs":{"total":96636764160,"available":96476569600,"internally_reserved":0,"allocated":76550144,"data_stored":76214844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":83619502},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416740,"kb_used_data":272660,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93955100,"statfs":{"total":96636764160,"available":96210022400,"internally_reserved":0,"allocated":279203840,"data_stored":278945078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,0,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":279023616,"data_stored":279020137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":76787712,"data_stored":76707695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278487040,"data_stored":278483561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:17.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:50:17.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'4731", 'reported_seq': 4804, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:12.195941+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:50:12.195941+0000', 'last_peered': '2024-09-16T04:50:12.195941+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:50:12.195941+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 4731, 'log_dups_size': 0, 'ondisk_log_size': 4731, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19378176, 'num_objects': 4731, 'num_object_clones': 0, 'num_object_copies': 9462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4731, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4731, 'num_write_kb': 18924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1390}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:17.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:50:17.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'4710", 'reported_seq': 4843, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:11.680707+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:50:11.680707+0000', 'last_peered': '2024-09-16T04:50:11.680707+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:50:11.680707+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 4710, 'log_dups_size': 0, 'ondisk_log_size': 4710, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19275776, 'num_objects': 4706, 'num_object_clones': 0, 'num_object_copies': 9412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4706, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1449, 'num_write_kb': 5796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 1387}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:17.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:50:17.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'4845", 'reported_seq': 4971, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:12.148682+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:50:12.148682+0000', 'last_peered': '2024-09-16T04:50:12.148682+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:50:12.148682+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 4845, 'log_dups_size': 0, 'ondisk_log_size': 4845, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19824640, 'num_objects': 4840, 'num_object_clones': 0, 'num_object_copies': 9680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4840, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1838, 'num_write_kb': 7352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1384}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:17.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:50:17.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'4776", 'reported_seq': 4919, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:12.199464+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:50:12.199464+0000', 'last_peered': '2024-09-16T04:50:12.199464+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:50:12.199464+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 4776, 'log_dups_size': 0, 'ondisk_log_size': 4776, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19537920, 'num_objects': 4770, 'num_object_clones': 0, 'num_object_copies': 9540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1431, 'num_write_kb': 5724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1371}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:17.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:50:17.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'4755", 'reported_seq': 4978, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:11.548416+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:50:11.548416+0000', 'last_peered': '2024-09-16T04:50:11.548416+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:50:11.548416+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 4755, 'log_dups_size': 0, 'ondisk_log_size': 4755, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19341312, 'num_objects': 4722, 'num_object_clones': 0, 'num_object_copies': 9444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4722, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1506, 'num_write_kb': 6024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 1401}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:17.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:50:17.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'4861", 'reported_seq': 4986, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:12.090057+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:50:12.090057+0000', 'last_peered': '2024-09-16T04:50:12.090057+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:50:12.090057+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 4861, 'log_dups_size': 0, 'ondisk_log_size': 4861, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19877888, 'num_objects': 4853, 'num_object_clones': 0, 'num_object_copies': 9706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1516, 'num_write_kb': 6064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1388}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:17.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:50:17.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'4729", 'reported_seq': 4966, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:11.664464+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:50:11.664464+0000', 'last_peered': '2024-09-16T04:50:11.664464+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:50:11.664464+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 4729, 'log_dups_size': 0, 'ondisk_log_size': 4729, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19244242, 'num_objects': 4700, 'num_object_clones': 0, 'num_object_copies': 9400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4700, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4701, 'num_write_kb': 18795, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 1404}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:17.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:50:17.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'4740", 'reported_seq': 6319, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:50:11.668156+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:50:11.668156+0000', 'last_peered': '2024-09-16T04:50:11.668156+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:50:11.668156+0000', 'last_undegraded': '2024-09-16T04:50:11.668156+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 4740, 'log_dups_size': 0, 'ondisk_log_size': 4740, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19390464, 'num_objects': 4734, 'num_object_clones': 0, 'num_object_copies': 9468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2630, 'num_objects_unfound': 0, 'num_objects_dirty': 4734, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1463, 'num_write_kb': 5852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 714, 'num_bytes_recovered': 2924544, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2630}, {'shards': '2', 'objects': 2104}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:17.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:50:17.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'4775", 'reported_seq': 4868, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:11.670152+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:50:11.670152+0000', 'last_peered': '2024-09-16T04:50:11.670152+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:50:11.670152+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 4775, 'log_dups_size': 0, 'ondisk_log_size': 4775, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19550825, 'num_objects': 4774, 'num_object_clones': 0, 'num_object_copies': 9548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4774, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4774, 'num_write_kb': 19093, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 1367}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:17.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:50:17.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'4849", 'reported_seq': 4947, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:11.642254+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:50:11.642254+0000', 'last_peered': '2024-09-16T04:50:11.642254+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:50:11.642254+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 4849, 'log_dups_size': 0, 'ondisk_log_size': 4849, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19830587, 'num_objects': 4844, 'num_object_clones': 0, 'num_object_copies': 9688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4844, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4846, 'num_write_kb': 19368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 1330}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:17.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:50:17.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'4726", 'reported_seq': 4852, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:12.151506+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:50:12.151506+0000', 'last_peered': '2024-09-16T04:50:12.151506+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:50:12.151506+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 4726, 'log_dups_size': 0, 'ondisk_log_size': 4726, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19341312, 'num_objects': 4722, 'num_object_clones': 0, 'num_object_copies': 9444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4722, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1480, 'num_write_kb': 5920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1397}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:17.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:50:17.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'4922", 'reported_seq': 5057, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:12.189274+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:50:12.189274+0000', 'last_peered': '2024-09-16T04:50:12.189274+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:50:12.189274+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 4922, 'log_dups_size': 0, 'ondisk_log_size': 4922, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20111977, 'num_objects': 4911, 'num_object_clones': 0, 'num_object_copies': 9822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4911, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1691, 'num_write_kb': 6764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1387}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:17.537 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:50:18.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 81 20 80092 80072 3.86055 2.23438 0.0227561 0.0202287 2024-09-16T04:50:18.453 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:18.453 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/158888 objects degraded (23.530%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":166,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":79444,"data_bytes":325370098,"bytes_used":1029103616,"bytes_avail":288881188864,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":158888,"degraded_ratio":0.23530411358944664,"misplaced_objects":2516,"misplaced_total":158888,"misplaced_ratio":0.015835053622677609,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":113626,"recovering_keys_per_sec":0,"num_objects_recovered":333,"num_bytes_recovered":1363968,"num_keys_recovered":0,"write_bytes_sec":3229660,"read_op_per_sec":0,"write_op_per_sec":788},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-16T04:49:42.278701+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T04:50:18.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:50:18.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:19.065 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 82 20 80677 80657 3.84134 2.28516 0.00889573 0.0203284 2024-09-16T04:50:19.352 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:19.352 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:19.425 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-16T04:50:18.292131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":325370098,"num_objects":79444,"num_object_clones":0,"num_object_copies":158888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2516,"num_objects_unfound":0,"num_objects_dirty":79444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44678,"num_write_kb":178678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":838,"num_bytes_recovered":3403795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79601,"ondisk_log_size":79601,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1004984,"kb_used_data":635076,"kb_used_omap":12,"kb_used_meta":369843,"kb_avail":282110536,"statfs":{"total":289910292480,"available":288881188864,"internally_reserved":0,"allocated":650317824,"data_stored":649465000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":378719233},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,13,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38768640,"num_objects":9465,"num_object_clones":0,"num_object_copies":18930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-333,"num_objects_unfound":0,"num_objects_dirty":9465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9465,"num_write_kb":37860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":1363968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003935"},"pg_stats":[{"pgid":"2.f","version":"45'4914","reported_seq":4988,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:17.162944+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:17.162944+0000","last_peered":"2024-09-16T04:50:17.162944+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:17.162944+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4914,"log_dups_size":0,"ondisk_log_size":4914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20127744,"num_objects":4914,"num_object_clones":0,"num_object_copies":9828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4914,"num_write_kb":19656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1573},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'4960","reported_seq":5071,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:16.681716+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:16.681716+0000","last_peered":"2024-09-16T04:50:16.681716+0000","last_clean":"2024-09-16T04:50:16.681716+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:16.681716+0000","last_undegraded":"2024-09-16T04:50:16.681716+0000","last_fullsized":"2024-09-16T04:50:16.681716+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":4960,"log_dups_size":0,"ondisk_log_size":4960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20291584,"num_objects":4954,"num_object_clones":0,"num_object_copies":9908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1929,"num_write_kb":7716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'4899","reported_seq":5031,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:16.709017+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:16.709017+0000","last_peered":"2024-09-16T04:50:16.709017+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:16.709017+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":4899,"log_dups_size":0,"ondisk_log_size":4899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20049920,"num_objects":4895,"num_object_clones":0,"num_object_copies":9790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1638,"num_write_kb":6552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":1576}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'4946","reported_seq":5011,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:16.691894+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:16.691894+0000","last_peered":"2024-09-16T04:50:16.691894+0000","last_clean":"2024-09-16T04:50:16.691894+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:16.691894+0000","last_undegraded":"2024-09-16T04:50:16.691894+0000","last_fullsized":"2024-09-16T04:50:16.691894+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4946,"log_dups_size":0,"ondisk_log_size":4946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20258816,"num_objects":4946,"num_object_clones":0,"num_object_copies":9892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4946,"num_write_kb":19784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'5022","reported_seq":5236,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:17.171540+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:17.171540+0000","last_peered":"2024-09-16T04:50:17.171540+0000","last_clean":"2024-09-16T04:50:17.171540+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:17.171540+0000","last_undegraded":"2024-09-16T04:50:17.171540+0000","last_fullsized":"2024-09-16T04:50:17.171540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":5022,"log_dups_size":0,"ondisk_log_size":5022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20447232,"num_objects":4992,"num_object_clones":0,"num_object_copies":9984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2054,"num_write_kb":8216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'5044","reported_seq":5172,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:17.133128+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:17.133128+0000","last_peered":"2024-09-16T04:50:17.133128+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:17.133128+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":5044,"log_dups_size":0,"ondisk_log_size":5044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20639744,"num_objects":5039,"num_object_clones":0,"num_object_copies":10078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2037,"num_write_kb":8148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1583},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'4965","reported_seq":5108,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:17.071417+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:17.071417+0000","last_peered":"2024-09-16T04:50:17.071417+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:17.071417+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":4965,"log_dups_size":0,"ondisk_log_size":4965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20312064,"num_objects":4959,"num_object_clones":0,"num_object_copies":9918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1620,"num_write_kb":6480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1560},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'4933","reported_seq":5144,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:17.084051+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:17.084051+0000","last_peered":"2024-09-16T04:50:17.084051+0000","last_clean":"2024-09-16T04:50:17.084051+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:17.084051+0000","last_undegraded":"2024-09-16T04:50:17.084051+0000","last_fullsized":"2024-09-16T04:50:17.084051+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":4933,"log_dups_size":0,"ondisk_log_size":4933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20172800,"num_objects":4925,"num_object_clones":0,"num_object_copies":9850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2003,"num_write_kb":8012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'4949","reported_seq":5170,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:16.702403+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:16.702403+0000","last_peered":"2024-09-16T04:50:16.702403+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:16.702403+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":4949,"log_dups_size":0,"ondisk_log_size":4949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20135936,"num_objects":4916,"num_object_clones":0,"num_object_copies":9832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1700,"num_write_kb":6800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":1595}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'5058","reported_seq":5183,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:17.116871+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:17.116871+0000","last_peered":"2024-09-16T04:50:17.116871+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:17.116871+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":5058,"log_dups_size":0,"ondisk_log_size":5058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20684800,"num_objects":5050,"num_object_clones":0,"num_object_copies":10100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1713,"num_write_kb":6852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1585},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'4929","reported_seq":5166,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:16.589646+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:16.589646+0000","last_peered":"2024-09-16T04:50:16.589646+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:16.589646+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":4929,"log_dups_size":0,"ondisk_log_size":4929,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20063442,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4901,"num_write_kb":19595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1604}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'4946","reported_seq":6750,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:16.733764+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:16.733764+0000","last_peered":"2024-09-16T04:50:16.733764+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:16.733764+0000","last_undegraded":"2024-09-16T04:50:16.733764+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":4946,"log_dups_size":0,"ondisk_log_size":4946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20234240,"num_objects":4940,"num_object_clones":0,"num_object_copies":9880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2516,"num_objects_unfound":0,"num_objects_dirty":4940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1669,"num_write_kb":6676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":828,"num_bytes_recovered":3391488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2516},{"shards":"2","objects":2424}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'4950","reported_seq":5043,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:16.681681+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:16.681681+0000","last_peered":"2024-09-16T04:50:16.681681+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:16.681681+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":4950,"log_dups_size":0,"ondisk_log_size":4950,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20267625,"num_objects":4949,"num_object_clones":0,"num_object_copies":9898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4949,"num_write_kb":19793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":1542}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'5049","reported_seq":5147,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:16.681795+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:16.681795+0000","last_peered":"2024-09-16T04:50:16.681795+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:16.681795+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":5049,"log_dups_size":0,"ondisk_log_size":5049,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20649787,"num_objects":5044,"num_object_clones":0,"num_object_copies":10088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5046,"num_write_kb":20168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":1530}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'4939","reported_seq":5065,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:17.153145+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:17.153145+0000","last_peered":"2024-09-16T04:50:17.153145+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:17.153145+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":4939,"log_dups_size":0,"ondisk_log_size":4939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20213760,"num_objects":4935,"num_object_clones":0,"num_object_copies":9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1693,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1610},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'5095","reported_seq":5230,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:17.180800+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:17.180800+0000","last_peered":"2024-09-16T04:50:17.180800+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:17.180800+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":5095,"log_dups_size":0,"ondisk_log_size":5095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20820585,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1864,"num_write_kb":7456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1560},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":325370079,"num_objects":79442,"num_object_clones":0,"num_object_copies":158884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2516,"num_objects_unfound":0,"num_objects_dirty":79442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44676,"num_write_kb":178676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":837,"num_bytes_recovered":3403776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":650149888,"data_stored":650062913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79598,"ondisk_log_size":79598,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415728,"kb_used_data":279904,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93956112,"statfs":{"total":96636764160,"available":96211058688,"internally_reserved":0,"allocated":286621696,"data_stored":286362934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":139062953},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,7,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593737,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":156440,"kb_used_data":74756,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94215400,"statfs":{"total":96636764160,"available":96476569600,"internally_reserved":0,"allocated":76550144,"data_stored":76214844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":83619502},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182428,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432816,"kb_used_data":280416,"kb_used_omap":4,"kb_used_meta":152379,"kb_avail":93939024,"statfs":{"total":96636764160,"available":96193560576,"internally_reserved":0,"allocated":287145984,"data_stored":286887222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":156036778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":286883840,"data_stored":286880361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":76787712,"data_stored":76707695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":286478336,"data_stored":286474857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:20.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 83 20 81302 81282 3.82447 2.44141 0.0443868 0.0204192 2024-09-16T04:50:21.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 84 20 81935 81915 3.80836 2.47266 0.0924193 0.0205039 2024-09-16T04:50:22.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 85 20 82551 82531 3.79187 2.40625 0.00590251 0.0205945 2024-09-16T04:50:22.428 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:23.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 86 20 83145 83125 3.77475 2.32031 0.0343714 0.0206853 2024-09-16T04:50:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:23.190 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:23.235 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-16T04:50:22.293011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":337834226,"num_objects":82487,"num_object_clones":0,"num_object_copies":164974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2401,"num_objects_unfound":0,"num_objects_dirty":82487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47721,"num_write_kb":190850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":953,"num_bytes_recovered":3874835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82644,"ondisk_log_size":82644,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1069724,"kb_used_data":659564,"kb_used_omap":12,"kb_used_meta":410099,"kb_avail":282045796,"statfs":{"total":289910292480,"available":288814895104,"internally_reserved":0,"allocated":675393536,"data_stored":674540712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":419941377},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,4,11,14,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38084608,"num_objects":9298,"num_object_clones":0,"num_object_copies":18596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-338,"num_objects_unfound":0,"num_objects_dirty":9298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9298,"num_write_kb":37192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":1384448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002743"},"pg_stats":[{"pgid":"2.f","version":"45'5098","reported_seq":5172,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.112049+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:22.112049+0000","last_peered":"2024-09-16T04:50:22.112049+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:22.112049+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5098,"log_dups_size":0,"ondisk_log_size":5098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20881408,"num_objects":5098,"num_object_clones":0,"num_object_copies":10196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5098,"num_write_kb":20392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1757},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'5155","reported_seq":5266,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:21.704080+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:21.704080+0000","last_peered":"2024-09-16T04:50:21.704080+0000","last_clean":"2024-09-16T04:50:21.704080+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:21.704080+0000","last_undegraded":"2024-09-16T04:50:21.704080+0000","last_fullsized":"2024-09-16T04:50:21.704080+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":5155,"log_dups_size":0,"ondisk_log_size":5155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21090304,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2124,"num_write_kb":8496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'5088","reported_seq":5221,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.737858+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:21.737858+0000","last_peered":"2024-09-16T04:50:21.737858+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:21.737858+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":5088,"log_dups_size":0,"ondisk_log_size":5088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1827,"num_write_kb":7308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":1765}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'5152","reported_seq":5217,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:21.666260+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:21.666260+0000","last_peered":"2024-09-16T04:50:21.666260+0000","last_clean":"2024-09-16T04:50:21.666260+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:21.666260+0000","last_undegraded":"2024-09-16T04:50:21.666260+0000","last_fullsized":"2024-09-16T04:50:21.666260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5152,"log_dups_size":0,"ondisk_log_size":5152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":5152,"num_object_clones":0,"num_object_copies":10304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5152,"num_write_kb":20608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'5197","reported_seq":5411,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:22.158086+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:22.158086+0000","last_peered":"2024-09-16T04:50:22.158086+0000","last_clean":"2024-09-16T04:50:22.158086+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:22.158086+0000","last_undegraded":"2024-09-16T04:50:22.158086+0000","last_fullsized":"2024-09-16T04:50:22.158086+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":5197,"log_dups_size":0,"ondisk_log_size":5197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21164032,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2229,"num_write_kb":8916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'5244","reported_seq":5371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.185139+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:22.185139+0000","last_peered":"2024-09-16T04:50:22.185139+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:22.185139+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":5244,"log_dups_size":0,"ondisk_log_size":5244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21458944,"num_objects":5239,"num_object_clones":0,"num_object_copies":10478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2237,"num_write_kb":8948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1783},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'5128","reported_seq":5271,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.128121+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:22.128121+0000","last_peered":"2024-09-16T04:50:22.128121+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:22.128121+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":5128,"log_dups_size":0,"ondisk_log_size":5128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20979712,"num_objects":5122,"num_object_clones":0,"num_object_copies":10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":7132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1723},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'5120","reported_seq":5331,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:22.206852+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:22.206852+0000","last_peered":"2024-09-16T04:50:22.206852+0000","last_clean":"2024-09-16T04:50:22.206852+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:22.206852+0000","last_undegraded":"2024-09-16T04:50:22.206852+0000","last_fullsized":"2024-09-16T04:50:22.206852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":5120,"log_dups_size":0,"ondisk_log_size":5120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20938752,"num_objects":5112,"num_object_clones":0,"num_object_copies":10224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2190,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'5140","reported_seq":5363,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.678794+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:21.678794+0000","last_peered":"2024-09-16T04:50:21.678794+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:21.678794+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":5140,"log_dups_size":0,"ondisk_log_size":5140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20918272,"num_objects":5107,"num_object_clones":0,"num_object_copies":10214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1891,"num_write_kb":7564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":1786}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'5239","reported_seq":5364,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.080736+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:22.080736+0000","last_peered":"2024-09-16T04:50:22.080736+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:22.080736+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21426176,"num_objects":5231,"num_object_clones":0,"num_object_copies":10462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1894,"num_write_kb":7576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1766},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'5112","reported_seq":5349,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.682601+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:21.682601+0000","last_peered":"2024-09-16T04:50:21.682601+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:21.682601+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":5112,"log_dups_size":0,"ondisk_log_size":5112,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20813010,"num_objects":5083,"num_object_clones":0,"num_object_copies":10166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5084,"num_write_kb":20327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1787}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'5133","reported_seq":7172,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:21.728059+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:21.728059+0000","last_peered":"2024-09-16T04:50:21.728059+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:21.728059+0000","last_undegraded":"2024-09-16T04:50:21.728059+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":5133,"log_dups_size":0,"ondisk_log_size":5133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21000192,"num_objects":5127,"num_object_clones":0,"num_object_copies":10254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2401,"num_objects_unfound":0,"num_objects_dirty":5127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1856,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":3862528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2401},{"shards":"2","objects":2726}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'5153","reported_seq":5246,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.693853+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:21.693853+0000","last_peered":"2024-09-16T04:50:21.693853+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:21.693853+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5153,"log_dups_size":0,"ondisk_log_size":5153,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21099113,"num_objects":5152,"num_object_clones":0,"num_object_copies":10304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5152,"num_write_kb":20605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":1745}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'5230","reported_seq":5328,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.712043+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:21.712043+0000","last_peered":"2024-09-16T04:50:21.712043+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:21.712043+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":5230,"log_dups_size":0,"ondisk_log_size":5230,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21391163,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5227,"num_write_kb":20892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":1711}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'5146","reported_seq":5271,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.082970+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:22.082970+0000","last_peered":"2024-09-16T04:50:22.082970+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:22.082970+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":5146,"log_dups_size":0,"ondisk_log_size":5146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21061632,"num_objects":5142,"num_object_clones":0,"num_object_copies":10284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1900,"num_write_kb":7600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1817},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'5306","reported_seq":5441,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.165119+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:22.165119+0000","last_peered":"2024-09-16T04:50:22.165119+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:22.165119+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":5306,"log_dups_size":0,"ondisk_log_size":5306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21684841,"num_objects":5295,"num_object_clones":0,"num_object_copies":10590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2075,"num_write_kb":8300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1771},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":337834207,"num_objects":82485,"num_object_clones":0,"num_object_copies":164970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2401,"num_objects_unfound":0,"num_objects_dirty":82485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47719,"num_write_kb":190848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":952,"num_bytes_recovered":3874816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":675487744,"data_stored":675400769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82641,"ondisk_log_size":82641,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":449660,"kb_used_data":287468,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93922180,"statfs":{"total":96636764160,"available":96176312320,"internally_reserved":0,"allocated":294367232,"data_stored":294108470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":166063785},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,7,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593738,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":174072,"kb_used_data":84072,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94197768,"statfs":{"total":96636764160,"available":96458514432,"internally_reserved":0,"allocated":86089728,"data_stored":85754428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92139182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445992,"kb_used_data":288024,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93925848,"statfs":{"total":96636764160,"available":96180068352,"internally_reserved":0,"allocated":294936576,"data_stored":294677814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,2,1,6,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":294772736,"data_stored":294769257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86495232,"data_stored":86415215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":294219776,"data_stored":294216297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:23.236 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:24.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 87 20 83741 83721 3.75813 2.32812 0.00782648 0.0207787 2024-09-16T04:50:24.226 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:24.228 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:24.291 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-16T04:50:22.293011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":337834226,"num_objects":82487,"num_object_clones":0,"num_object_copies":164974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2401,"num_objects_unfound":0,"num_objects_dirty":82487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47721,"num_write_kb":190850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":953,"num_bytes_recovered":3874835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82644,"ondisk_log_size":82644,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1069724,"kb_used_data":659564,"kb_used_omap":12,"kb_used_meta":410099,"kb_avail":282045796,"statfs":{"total":289910292480,"available":288814895104,"internally_reserved":0,"allocated":675393536,"data_stored":674540712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":419941377},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,4,11,14,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38084608,"num_objects":9298,"num_object_clones":0,"num_object_copies":18596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-338,"num_objects_unfound":0,"num_objects_dirty":9298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9298,"num_write_kb":37192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":1384448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002743"},"pg_stats":[{"pgid":"2.f","version":"45'5098","reported_seq":5172,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.112049+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:22.112049+0000","last_peered":"2024-09-16T04:50:22.112049+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:22.112049+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5098,"log_dups_size":0,"ondisk_log_size":5098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20881408,"num_objects":5098,"num_object_clones":0,"num_object_copies":10196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5098,"num_write_kb":20392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1757},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'5155","reported_seq":5266,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:21.704080+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:21.704080+0000","last_peered":"2024-09-16T04:50:21.704080+0000","last_clean":"2024-09-16T04:50:21.704080+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:21.704080+0000","last_undegraded":"2024-09-16T04:50:21.704080+0000","last_fullsized":"2024-09-16T04:50:21.704080+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":5155,"log_dups_size":0,"ondisk_log_size":5155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21090304,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2124,"num_write_kb":8496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'5088","reported_seq":5221,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.737858+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:21.737858+0000","last_peered":"2024-09-16T04:50:21.737858+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:21.737858+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":5088,"log_dups_size":0,"ondisk_log_size":5088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1827,"num_write_kb":7308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":1765}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'5152","reported_seq":5217,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:21.666260+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:21.666260+0000","last_peered":"2024-09-16T04:50:21.666260+0000","last_clean":"2024-09-16T04:50:21.666260+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:21.666260+0000","last_undegraded":"2024-09-16T04:50:21.666260+0000","last_fullsized":"2024-09-16T04:50:21.666260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5152,"log_dups_size":0,"ondisk_log_size":5152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":5152,"num_object_clones":0,"num_object_copies":10304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5152,"num_write_kb":20608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'5197","reported_seq":5411,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:22.158086+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:22.158086+0000","last_peered":"2024-09-16T04:50:22.158086+0000","last_clean":"2024-09-16T04:50:22.158086+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:22.158086+0000","last_undegraded":"2024-09-16T04:50:22.158086+0000","last_fullsized":"2024-09-16T04:50:22.158086+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":5197,"log_dups_size":0,"ondisk_log_size":5197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21164032,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2229,"num_write_kb":8916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'5244","reported_seq":5371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.185139+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:22.185139+0000","last_peered":"2024-09-16T04:50:22.185139+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:22.185139+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":5244,"log_dups_size":0,"ondisk_log_size":5244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21458944,"num_objects":5239,"num_object_clones":0,"num_object_copies":10478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2237,"num_write_kb":8948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1783},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'5128","reported_seq":5271,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.128121+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:22.128121+0000","last_peered":"2024-09-16T04:50:22.128121+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:22.128121+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":5128,"log_dups_size":0,"ondisk_log_size":5128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20979712,"num_objects":5122,"num_object_clones":0,"num_object_copies":10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":7132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1723},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'5120","reported_seq":5331,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:22.206852+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:22.206852+0000","last_peered":"2024-09-16T04:50:22.206852+0000","last_clean":"2024-09-16T04:50:22.206852+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:22.206852+0000","last_undegraded":"2024-09-16T04:50:22.206852+0000","last_fullsized":"2024-09-16T04:50:22.206852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":5120,"log_dups_size":0,"ondisk_log_size":5120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20938752,"num_objects":5112,"num_object_clones":0,"num_object_copies":10224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2190,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'5140","reported_seq":5363,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.678794+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:21.678794+0000","last_peered":"2024-09-16T04:50:21.678794+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:21.678794+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":5140,"log_dups_size":0,"ondisk_log_size":5140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20918272,"num_objects":5107,"num_object_clones":0,"num_object_copies":10214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1891,"num_write_kb":7564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":1786}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'5239","reported_seq":5364,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.080736+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:22.080736+0000","last_peered":"2024-09-16T04:50:22.080736+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:22.080736+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21426176,"num_objects":5231,"num_object_clones":0,"num_object_copies":10462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1894,"num_write_kb":7576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1766},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'5112","reported_seq":5349,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.682601+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:21.682601+0000","last_peered":"2024-09-16T04:50:21.682601+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:21.682601+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":5112,"log_dups_size":0,"ondisk_log_size":5112,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20813010,"num_objects":5083,"num_object_clones":0,"num_object_copies":10166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5084,"num_write_kb":20327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1787}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'5133","reported_seq":7172,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:21.728059+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:21.728059+0000","last_peered":"2024-09-16T04:50:21.728059+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:21.728059+0000","last_undegraded":"2024-09-16T04:50:21.728059+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":5133,"log_dups_size":0,"ondisk_log_size":5133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21000192,"num_objects":5127,"num_object_clones":0,"num_object_copies":10254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2401,"num_objects_unfound":0,"num_objects_dirty":5127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1856,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":3862528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2401},{"shards":"2","objects":2726}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'5153","reported_seq":5246,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.693853+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:21.693853+0000","last_peered":"2024-09-16T04:50:21.693853+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:21.693853+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5153,"log_dups_size":0,"ondisk_log_size":5153,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21099113,"num_objects":5152,"num_object_clones":0,"num_object_copies":10304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5152,"num_write_kb":20605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":1745}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'5230","reported_seq":5328,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.712043+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:21.712043+0000","last_peered":"2024-09-16T04:50:21.712043+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:21.712043+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":5230,"log_dups_size":0,"ondisk_log_size":5230,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21391163,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5227,"num_write_kb":20892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":1711}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'5146","reported_seq":5271,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.082970+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:22.082970+0000","last_peered":"2024-09-16T04:50:22.082970+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:22.082970+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":5146,"log_dups_size":0,"ondisk_log_size":5146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21061632,"num_objects":5142,"num_object_clones":0,"num_object_copies":10284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1900,"num_write_kb":7600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1817},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'5306","reported_seq":5441,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.165119+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:22.165119+0000","last_peered":"2024-09-16T04:50:22.165119+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:22.165119+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":5306,"log_dups_size":0,"ondisk_log_size":5306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21684841,"num_objects":5295,"num_object_clones":0,"num_object_copies":10590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2075,"num_write_kb":8300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1771},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":337834207,"num_objects":82485,"num_object_clones":0,"num_object_copies":164970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2401,"num_objects_unfound":0,"num_objects_dirty":82485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47719,"num_write_kb":190848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":952,"num_bytes_recovered":3874816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":675487744,"data_stored":675400769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82641,"ondisk_log_size":82641,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":449660,"kb_used_data":287468,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93922180,"statfs":{"total":96636764160,"available":96176312320,"internally_reserved":0,"allocated":294367232,"data_stored":294108470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":166063785},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,7,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593738,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":174072,"kb_used_data":84072,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94197768,"statfs":{"total":96636764160,"available":96458514432,"internally_reserved":0,"allocated":86089728,"data_stored":85754428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92139182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445992,"kb_used_data":288024,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93925848,"statfs":{"total":96636764160,"available":96180068352,"internally_reserved":0,"allocated":294936576,"data_stored":294677814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,2,1,6,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":294772736,"data_stored":294769257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86495232,"data_stored":86415215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":294219776,"data_stored":294216297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:24.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:50:24.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'5098", 'reported_seq': 5172, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:22.112049+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:50:22.112049+0000', 'last_peered': '2024-09-16T04:50:22.112049+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:50:22.112049+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 5098, 'log_dups_size': 0, 'ondisk_log_size': 5098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20881408, 'num_objects': 5098, 'num_object_clones': 0, 'num_object_copies': 10196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5098, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5098, 'num_write_kb': 20392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1757}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:24.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:50:24.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'5088", 'reported_seq': 5221, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:21.737858+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:50:21.737858+0000', 'last_peered': '2024-09-16T04:50:21.737858+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:50:21.737858+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 5088, 'log_dups_size': 0, 'ondisk_log_size': 5088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20824064, 'num_objects': 5084, 'num_object_clones': 0, 'num_object_copies': 10168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5084, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1827, 'num_write_kb': 7308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 1765}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:24.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:50:24.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'5244", 'reported_seq': 5371, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:22.185139+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:50:22.185139+0000', 'last_peered': '2024-09-16T04:50:22.185139+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:50:22.185139+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 5244, 'log_dups_size': 0, 'ondisk_log_size': 5244, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21458944, 'num_objects': 5239, 'num_object_clones': 0, 'num_object_copies': 10478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2237, 'num_write_kb': 8948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1783}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:24.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:50:24.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'5128", 'reported_seq': 5271, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:22.128121+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:50:22.128121+0000', 'last_peered': '2024-09-16T04:50:22.128121+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:50:22.128121+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 5128, 'log_dups_size': 0, 'ondisk_log_size': 5128, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20979712, 'num_objects': 5122, 'num_object_clones': 0, 'num_object_copies': 10244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5122, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1783, 'num_write_kb': 7132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1723}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:24.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:50:24.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'5140", 'reported_seq': 5363, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:21.678794+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:50:21.678794+0000', 'last_peered': '2024-09-16T04:50:21.678794+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:50:21.678794+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 5140, 'log_dups_size': 0, 'ondisk_log_size': 5140, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20918272, 'num_objects': 5107, 'num_object_clones': 0, 'num_object_copies': 10214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5107, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1891, 'num_write_kb': 7564, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 1786}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:24.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:50:24.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'5239", 'reported_seq': 5364, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:22.080736+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:50:22.080736+0000', 'last_peered': '2024-09-16T04:50:22.080736+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:50:22.080736+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 5239, 'log_dups_size': 0, 'ondisk_log_size': 5239, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21426176, 'num_objects': 5231, 'num_object_clones': 0, 'num_object_copies': 10462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5231, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1894, 'num_write_kb': 7576, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1766}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:24.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:50:24.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'5112", 'reported_seq': 5349, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:21.682601+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:50:21.682601+0000', 'last_peered': '2024-09-16T04:50:21.682601+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:50:21.682601+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 5112, 'log_dups_size': 0, 'ondisk_log_size': 5112, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20813010, 'num_objects': 5083, 'num_object_clones': 0, 'num_object_copies': 10166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5083, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5084, 'num_write_kb': 20327, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 1787}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:24.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:50:24.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'5133", 'reported_seq': 7172, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:50:21.728059+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:50:21.728059+0000', 'last_peered': '2024-09-16T04:50:21.728059+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:50:21.728059+0000', 'last_undegraded': '2024-09-16T04:50:21.728059+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 5133, 'log_dups_size': 0, 'ondisk_log_size': 5133, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21000192, 'num_objects': 5127, 'num_object_clones': 0, 'num_object_copies': 10254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2401, 'num_objects_unfound': 0, 'num_objects_dirty': 5127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1856, 'num_write_kb': 7424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 943, 'num_bytes_recovered': 3862528, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2401}, {'shards': '2', 'objects': 2726}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:24.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:50:24.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'5153", 'reported_seq': 5246, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:21.693853+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:50:21.693853+0000', 'last_peered': '2024-09-16T04:50:21.693853+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:50:21.693853+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 5153, 'log_dups_size': 0, 'ondisk_log_size': 5153, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21099113, 'num_objects': 5152, 'num_object_clones': 0, 'num_object_copies': 10304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5152, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5152, 'num_write_kb': 20605, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 1745}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:24.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:50:24.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'5230", 'reported_seq': 5328, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:21.712043+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:50:21.712043+0000', 'last_peered': '2024-09-16T04:50:21.712043+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:50:21.712043+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 5230, 'log_dups_size': 0, 'ondisk_log_size': 5230, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21391163, 'num_objects': 5225, 'num_object_clones': 0, 'num_object_copies': 10450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5225, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5227, 'num_write_kb': 20892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 1711}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:24.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:50:24.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'5146", 'reported_seq': 5271, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:22.082970+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:50:22.082970+0000', 'last_peered': '2024-09-16T04:50:22.082970+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:50:22.082970+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 5146, 'log_dups_size': 0, 'ondisk_log_size': 5146, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21061632, 'num_objects': 5142, 'num_object_clones': 0, 'num_object_copies': 10284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5142, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1900, 'num_write_kb': 7600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1817}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:24.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:50:24.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'5306", 'reported_seq': 5441, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:22.165119+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:50:22.165119+0000', 'last_peered': '2024-09-16T04:50:22.165119+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:50:22.165119+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 5306, 'log_dups_size': 0, 'ondisk_log_size': 5306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21684841, 'num_objects': 5295, 'num_object_clones': 0, 'num_object_copies': 10590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5295, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2075, 'num_write_kb': 8300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1771}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:24.299 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:50:25.066 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 88 20 84327 84307 3.74143 2.28906 0.0488362 0.0208712 2024-09-16T04:50:25.261 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:25.261 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/164974 objects degraded (22.662%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":173,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":82487,"data_bytes":337834226,"bytes_used":1095397376,"bytes_avail":288814895104,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":164974,"degraded_ratio":0.22662358917162703,"misplaced_objects":2401,"misplaced_total":164974,"misplaced_ratio":0.01455380847891183,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":78147,"recovering_keys_per_sec":0,"num_objects_recovered":229,"num_bytes_recovered":937984,"num_keys_recovered":0,"write_bytes_sec":2091545,"read_op_per_sec":0,"write_op_per_sec":510},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-16T04:49:42.278701+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T04:50:25.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:50:25.310 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:26.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 89 20 84898 84878 3.72442 2.23047 0.00951155 0.0209648 2024-09-16T04:50:26.113 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:26.114 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:26.160 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-09-16T04:50:24.293472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":337834226,"num_objects":82487,"num_object_clones":0,"num_object_copies":164974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2401,"num_objects_unfound":0,"num_objects_dirty":82487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47721,"num_write_kb":190850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":953,"num_bytes_recovered":3874835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82644,"ondisk_log_size":82644,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1069724,"kb_used_data":659564,"kb_used_omap":12,"kb_used_meta":410099,"kb_avail":282045796,"statfs":{"total":289910292480,"available":288814895104,"internally_reserved":0,"allocated":675393536,"data_stored":674540712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":419941377},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,4,11,14,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25104384,"num_objects":6129,"num_object_clones":0,"num_object_copies":12258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-229,"num_objects_unfound":0,"num_objects_dirty":6129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6129,"num_write_kb":24516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":937984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002789"},"pg_stats":[{"pgid":"2.f","version":"45'5098","reported_seq":5172,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.112049+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:22.112049+0000","last_peered":"2024-09-16T04:50:22.112049+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:22.112049+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5098,"log_dups_size":0,"ondisk_log_size":5098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20881408,"num_objects":5098,"num_object_clones":0,"num_object_copies":10196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5098,"num_write_kb":20392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1757},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'5155","reported_seq":5266,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:21.704080+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:21.704080+0000","last_peered":"2024-09-16T04:50:21.704080+0000","last_clean":"2024-09-16T04:50:21.704080+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:21.704080+0000","last_undegraded":"2024-09-16T04:50:21.704080+0000","last_fullsized":"2024-09-16T04:50:21.704080+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":5155,"log_dups_size":0,"ondisk_log_size":5155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21090304,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2124,"num_write_kb":8496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'5088","reported_seq":5221,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.737858+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:21.737858+0000","last_peered":"2024-09-16T04:50:21.737858+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:21.737858+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":5088,"log_dups_size":0,"ondisk_log_size":5088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1827,"num_write_kb":7308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":1765}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'5152","reported_seq":5217,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:21.666260+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:21.666260+0000","last_peered":"2024-09-16T04:50:21.666260+0000","last_clean":"2024-09-16T04:50:21.666260+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:21.666260+0000","last_undegraded":"2024-09-16T04:50:21.666260+0000","last_fullsized":"2024-09-16T04:50:21.666260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5152,"log_dups_size":0,"ondisk_log_size":5152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":5152,"num_object_clones":0,"num_object_copies":10304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5152,"num_write_kb":20608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'5197","reported_seq":5411,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:22.158086+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:22.158086+0000","last_peered":"2024-09-16T04:50:22.158086+0000","last_clean":"2024-09-16T04:50:22.158086+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:22.158086+0000","last_undegraded":"2024-09-16T04:50:22.158086+0000","last_fullsized":"2024-09-16T04:50:22.158086+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":5197,"log_dups_size":0,"ondisk_log_size":5197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21164032,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2229,"num_write_kb":8916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'5244","reported_seq":5371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.185139+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:22.185139+0000","last_peered":"2024-09-16T04:50:22.185139+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:22.185139+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":5244,"log_dups_size":0,"ondisk_log_size":5244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21458944,"num_objects":5239,"num_object_clones":0,"num_object_copies":10478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2237,"num_write_kb":8948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1783},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'5128","reported_seq":5271,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.128121+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:22.128121+0000","last_peered":"2024-09-16T04:50:22.128121+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:22.128121+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":5128,"log_dups_size":0,"ondisk_log_size":5128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20979712,"num_objects":5122,"num_object_clones":0,"num_object_copies":10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1783,"num_write_kb":7132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1723},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'5120","reported_seq":5331,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:22.206852+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:22.206852+0000","last_peered":"2024-09-16T04:50:22.206852+0000","last_clean":"2024-09-16T04:50:22.206852+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:22.206852+0000","last_undegraded":"2024-09-16T04:50:22.206852+0000","last_fullsized":"2024-09-16T04:50:22.206852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":5120,"log_dups_size":0,"ondisk_log_size":5120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20938752,"num_objects":5112,"num_object_clones":0,"num_object_copies":10224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2190,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'5140","reported_seq":5363,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.678794+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:21.678794+0000","last_peered":"2024-09-16T04:50:21.678794+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:21.678794+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":5140,"log_dups_size":0,"ondisk_log_size":5140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20918272,"num_objects":5107,"num_object_clones":0,"num_object_copies":10214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1891,"num_write_kb":7564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":1786}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'5239","reported_seq":5364,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.080736+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:22.080736+0000","last_peered":"2024-09-16T04:50:22.080736+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:22.080736+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21426176,"num_objects":5231,"num_object_clones":0,"num_object_copies":10462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1894,"num_write_kb":7576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1766},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.232856+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.232856+0000","last_peered":"2024-09-16T04:49:38.232856+0000","last_clean":"2024-09-16T04:49:38.232856+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.232856+0000","last_undegraded":"2024-09-16T04:49:38.232856+0000","last_fullsized":"2024-09-16T04:49:38.232856+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'5112","reported_seq":5349,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.682601+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:21.682601+0000","last_peered":"2024-09-16T04:50:21.682601+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:21.682601+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":5112,"log_dups_size":0,"ondisk_log_size":5112,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20813010,"num_objects":5083,"num_object_clones":0,"num_object_copies":10166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5084,"num_write_kb":20327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1787}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'5133","reported_seq":7172,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:21.728059+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:21.728059+0000","last_peered":"2024-09-16T04:50:21.728059+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:21.728059+0000","last_undegraded":"2024-09-16T04:50:21.728059+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":5133,"log_dups_size":0,"ondisk_log_size":5133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21000192,"num_objects":5127,"num_object_clones":0,"num_object_copies":10254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2401,"num_objects_unfound":0,"num_objects_dirty":5127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1856,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":3862528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2401},{"shards":"2","objects":2726}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'5153","reported_seq":5246,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.693853+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:21.693853+0000","last_peered":"2024-09-16T04:50:21.693853+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:21.693853+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5153,"log_dups_size":0,"ondisk_log_size":5153,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21099113,"num_objects":5152,"num_object_clones":0,"num_object_copies":10304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5152,"num_write_kb":20605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":1745}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'5230","reported_seq":5328,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:21.712043+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:21.712043+0000","last_peered":"2024-09-16T04:50:21.712043+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:21.712043+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":5230,"log_dups_size":0,"ondisk_log_size":5230,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21391163,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5227,"num_write_kb":20892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":1711}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'5146","reported_seq":5271,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.082970+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:22.082970+0000","last_peered":"2024-09-16T04:50:22.082970+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:22.082970+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":5146,"log_dups_size":0,"ondisk_log_size":5146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21061632,"num_objects":5142,"num_object_clones":0,"num_object_copies":10284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1900,"num_write_kb":7600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1817},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'5306","reported_seq":5441,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:22.165119+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:22.165119+0000","last_peered":"2024-09-16T04:50:22.165119+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:22.165119+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":5306,"log_dups_size":0,"ondisk_log_size":5306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21684841,"num_objects":5295,"num_object_clones":0,"num_object_copies":10590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2075,"num_write_kb":8300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1771},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":337834207,"num_objects":82485,"num_object_clones":0,"num_object_copies":164970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2401,"num_objects_unfound":0,"num_objects_dirty":82485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47719,"num_write_kb":190848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":952,"num_bytes_recovered":3874816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":675487744,"data_stored":675400769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82641,"ondisk_log_size":82641,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":449660,"kb_used_data":287468,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93922180,"statfs":{"total":96636764160,"available":96176312320,"internally_reserved":0,"allocated":294367232,"data_stored":294108470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":166063785},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,7,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593738,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":174072,"kb_used_data":84072,"kb_used_omap":4,"kb_used_meta":89979,"kb_avail":94197768,"statfs":{"total":96636764160,"available":96458514432,"internally_reserved":0,"allocated":86089728,"data_stored":85754428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":92139182},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445992,"kb_used_data":288024,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93925848,"statfs":{"total":96636764160,"available":96180068352,"internally_reserved":0,"allocated":294936576,"data_stored":294677814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,2,1,6,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":294772736,"data_stored":294769257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86495232,"data_stored":86415215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":294219776,"data_stored":294216297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:27.067 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 90 20 85530 85510 3.71047 2.46875 0.037758 0.021048 2024-09-16T04:50:28.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 91 20 86159 86139 3.69669 2.45703 0.0193744 0.0211264 2024-09-16T04:50:29.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 92 20 86772 86752 3.68254 2.39453 0.0111788 0.0212055 2024-09-16T04:50:29.163 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:29.961 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:29.962 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:30.055 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-16T04:50:28.294307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":350068978,"num_objects":85474,"num_object_clones":0,"num_object_copies":170948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2285,"num_objects_unfound":0,"num_objects_dirty":85474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50708,"num_write_kb":202798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1069,"num_bytes_recovered":4349971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85631,"ondisk_log_size":85631,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1112380,"kb_used_data":684172,"kb_used_omap":12,"kb_used_meta":428147,"kb_avail":282003140,"statfs":{"total":289910292480,"available":288771215360,"internally_reserved":0,"allocated":700592128,"data_stored":699739304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":438422529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,6,2,6,7,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":37339136,"num_objects":9116,"num_object_clones":0,"num_object_copies":18232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-345,"num_objects_unfound":0,"num_objects_dirty":9116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9116,"num_write_kb":36464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":1413120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002744"},"pg_stats":[{"pgid":"2.f","version":"45'5293","reported_seq":5367,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.218014+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:27.218014+0000","last_peered":"2024-09-16T04:50:27.218014+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:27.218014+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5293,"log_dups_size":0,"ondisk_log_size":5293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21680128,"num_objects":5293,"num_object_clones":0,"num_object_copies":10586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5293,"num_write_kb":21172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1952},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'5361","reported_seq":5472,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:26.739485+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:26.739485+0000","last_peered":"2024-09-16T04:50:26.739485+0000","last_clean":"2024-09-16T04:50:26.739485+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:26.739485+0000","last_undegraded":"2024-09-16T04:50:26.739485+0000","last_fullsized":"2024-09-16T04:50:26.739485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":5361,"log_dups_size":0,"ondisk_log_size":5361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21934080,"num_objects":5355,"num_object_clones":0,"num_object_copies":10710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2330,"num_write_kb":9320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'5262","reported_seq":5395,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:26.674240+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:26.674240+0000","last_peered":"2024-09-16T04:50:26.674240+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:26.674240+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":5262,"log_dups_size":0,"ondisk_log_size":5262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21536768,"num_objects":5258,"num_object_clones":0,"num_object_copies":10516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2001,"num_write_kb":8004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":1939}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'5336","reported_seq":5401,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:26.679207+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:26.679207+0000","last_peered":"2024-09-16T04:50:26.679207+0000","last_clean":"2024-09-16T04:50:26.679207+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:26.679207+0000","last_undegraded":"2024-09-16T04:50:26.679207+0000","last_fullsized":"2024-09-16T04:50:26.679207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21856256,"num_objects":5336,"num_object_clones":0,"num_object_copies":10672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5336,"num_write_kb":21344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'5375","reported_seq":5589,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:27.219123+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:27.219123+0000","last_peered":"2024-09-16T04:50:27.219123+0000","last_clean":"2024-09-16T04:50:27.219123+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:27.219123+0000","last_undegraded":"2024-09-16T04:50:27.219123+0000","last_fullsized":"2024-09-16T04:50:27.219123+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":5375,"log_dups_size":0,"ondisk_log_size":5375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21893120,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2407,"num_write_kb":9628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'5438","reported_seq":5566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.150266+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:27.150266+0000","last_peered":"2024-09-16T04:50:27.150266+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:27.150266+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":5438,"log_dups_size":0,"ondisk_log_size":5438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22253568,"num_objects":5433,"num_object_clones":0,"num_object_copies":10866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2431,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1977},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'5306","reported_seq":5449,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.186097+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:27.186097+0000","last_peered":"2024-09-16T04:50:27.186097+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:27.186097+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":5306,"log_dups_size":0,"ondisk_log_size":5306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21708800,"num_objects":5300,"num_object_clones":0,"num_object_copies":10600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1961,"num_write_kb":7844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1901},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'5292","reported_seq":5503,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:27.079033+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:27.079033+0000","last_peered":"2024-09-16T04:50:27.079033+0000","last_clean":"2024-09-16T04:50:27.079033+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:27.079033+0000","last_undegraded":"2024-09-16T04:50:27.079033+0000","last_fullsized":"2024-09-16T04:50:27.079033+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":5292,"log_dups_size":0,"ondisk_log_size":5292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21643264,"num_objects":5284,"num_object_clones":0,"num_object_copies":10568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2362,"num_write_kb":9448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'5319","reported_seq":5542,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:26.760450+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:26.760450+0000","last_peered":"2024-09-16T04:50:26.760450+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:26.760450+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":5319,"log_dups_size":0,"ondisk_log_size":5319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21651456,"num_objects":5286,"num_object_clones":0,"num_object_copies":10572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2070,"num_write_kb":8280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":1965}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'5432","reported_seq":5555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.194334+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:27.194334+0000","last_peered":"2024-09-16T04:50:27.194334+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:27.194334+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":5432,"log_dups_size":0,"ondisk_log_size":5432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22216704,"num_objects":5424,"num_object_clones":0,"num_object_copies":10848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2087,"num_write_kb":8348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1959},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'5297","reported_seq":5534,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:26.709004+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:26.709004+0000","last_peered":"2024-09-16T04:50:26.709004+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:26.709004+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":5297,"log_dups_size":0,"ondisk_log_size":5297,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21570770,"num_objects":5268,"num_object_clones":0,"num_object_copies":10536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5269,"num_write_kb":21067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'5337","reported_seq":7607,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:26.768053+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:26.768053+0000","last_peered":"2024-09-16T04:50:26.768053+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:26.768053+0000","last_undegraded":"2024-09-16T04:50:26.768053+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":5337,"log_dups_size":0,"ondisk_log_size":5337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21835776,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2285,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2060,"num_write_kb":8240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1059,"num_bytes_recovered":4337664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2285},{"shards":"2","objects":3046}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'5330","reported_seq":5422,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:26.713263+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:26.713263+0000","last_peered":"2024-09-16T04:50:26.713263+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:26.713263+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21824105,"num_objects":5329,"num_object_clones":0,"num_object_copies":10658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5329,"num_write_kb":21313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":1922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'5416","reported_seq":5514,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:26.763851+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:26.763851+0000","last_peered":"2024-09-16T04:50:26.763851+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:26.763851+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":5416,"log_dups_size":0,"ondisk_log_size":5416,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22153019,"num_objects":5411,"num_object_clones":0,"num_object_copies":10822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5413,"num_write_kb":21636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":1897}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'5337","reported_seq":5463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.185990+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:27.185990+0000","last_peered":"2024-09-16T04:50:27.185990+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:27.185990+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":5337,"log_dups_size":0,"ondisk_log_size":5337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21843968,"num_objects":5333,"num_object_clones":0,"num_object_copies":10666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2091,"num_write_kb":8364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2008},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'5497","reported_seq":5632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.202466+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:27.202466+0000","last_peered":"2024-09-16T04:50:27.202466+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:27.202466+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":5497,"log_dups_size":0,"ondisk_log_size":5497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22467177,"num_objects":5486,"num_object_clones":0,"num_object_copies":10972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2266,"num_write_kb":9064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1962},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":350068959,"num_objects":85472,"num_object_clones":0,"num_object_copies":170944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2285,"num_objects_unfound":0,"num_objects_dirty":85472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50706,"num_write_kb":202796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1068,"num_bytes_recovered":4349952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":700571648,"data_stored":700484673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85628,"ondisk_log_size":85628,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461244,"kb_used_data":294892,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93910596,"statfs":{"total":96636764160,"available":96164450304,"internally_reserved":0,"allocated":301969408,"data_stored":301710646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":170323625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593739,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":193588,"kb_used_data":93860,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94178252,"statfs":{"total":96636764160,"available":96438530048,"internally_reserved":0,"allocated":96112640,"data_stored":95777340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":102100654},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,2,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457548,"kb_used_data":295420,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93914292,"statfs":{"total":96636764160,"available":96168235008,"internally_reserved":0,"allocated":302510080,"data_stored":302251318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":302432256,"data_stored":302428777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96350208,"data_stored":96270191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":301789184,"data_stored":301785705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:30.057 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:30.067 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 93 20 87355 87335 3.66743 2.27734 0.0796029 0.0212934 2024-09-16T04:50:30.786 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:30.788 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:30.837 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-16T04:50:30.294753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":350068978,"num_objects":85474,"num_object_clones":0,"num_object_copies":170948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2285,"num_objects_unfound":0,"num_objects_dirty":85474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50708,"num_write_kb":202798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1069,"num_bytes_recovered":4349971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85631,"ondisk_log_size":85631,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1148356,"kb_used_data":693780,"kb_used_omap":12,"kb_used_meta":454515,"kb_avail":281967164,"statfs":{"total":289910292480,"available":288734375936,"internally_reserved":0,"allocated":710430720,"data_stored":709577896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":465423361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,4,6,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24698880,"num_objects":6030,"num_object_clones":0,"num_object_copies":12060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-231,"num_objects_unfound":0,"num_objects_dirty":6030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6030,"num_write_kb":24120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":946176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002621"},"pg_stats":[{"pgid":"2.f","version":"45'5293","reported_seq":5367,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.218014+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:27.218014+0000","last_peered":"2024-09-16T04:50:27.218014+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:27.218014+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5293,"log_dups_size":0,"ondisk_log_size":5293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21680128,"num_objects":5293,"num_object_clones":0,"num_object_copies":10586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5293,"num_write_kb":21172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1952},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'5361","reported_seq":5472,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:26.739485+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:26.739485+0000","last_peered":"2024-09-16T04:50:26.739485+0000","last_clean":"2024-09-16T04:50:26.739485+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:26.739485+0000","last_undegraded":"2024-09-16T04:50:26.739485+0000","last_fullsized":"2024-09-16T04:50:26.739485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":5361,"log_dups_size":0,"ondisk_log_size":5361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21934080,"num_objects":5355,"num_object_clones":0,"num_object_copies":10710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2330,"num_write_kb":9320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'5262","reported_seq":5395,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:26.674240+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:26.674240+0000","last_peered":"2024-09-16T04:50:26.674240+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:26.674240+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":5262,"log_dups_size":0,"ondisk_log_size":5262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21536768,"num_objects":5258,"num_object_clones":0,"num_object_copies":10516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2001,"num_write_kb":8004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":1939}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'5336","reported_seq":5401,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:26.679207+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:26.679207+0000","last_peered":"2024-09-16T04:50:26.679207+0000","last_clean":"2024-09-16T04:50:26.679207+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:26.679207+0000","last_undegraded":"2024-09-16T04:50:26.679207+0000","last_fullsized":"2024-09-16T04:50:26.679207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21856256,"num_objects":5336,"num_object_clones":0,"num_object_copies":10672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5336,"num_write_kb":21344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'5375","reported_seq":5589,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:27.219123+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:27.219123+0000","last_peered":"2024-09-16T04:50:27.219123+0000","last_clean":"2024-09-16T04:50:27.219123+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:27.219123+0000","last_undegraded":"2024-09-16T04:50:27.219123+0000","last_fullsized":"2024-09-16T04:50:27.219123+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":5375,"log_dups_size":0,"ondisk_log_size":5375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21893120,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2407,"num_write_kb":9628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'5438","reported_seq":5566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.150266+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:27.150266+0000","last_peered":"2024-09-16T04:50:27.150266+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:27.150266+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":5438,"log_dups_size":0,"ondisk_log_size":5438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22253568,"num_objects":5433,"num_object_clones":0,"num_object_copies":10866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2431,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1977},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'5306","reported_seq":5449,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.186097+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:27.186097+0000","last_peered":"2024-09-16T04:50:27.186097+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:27.186097+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":5306,"log_dups_size":0,"ondisk_log_size":5306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21708800,"num_objects":5300,"num_object_clones":0,"num_object_copies":10600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1961,"num_write_kb":7844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1901},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'5292","reported_seq":5503,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:27.079033+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:27.079033+0000","last_peered":"2024-09-16T04:50:27.079033+0000","last_clean":"2024-09-16T04:50:27.079033+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:27.079033+0000","last_undegraded":"2024-09-16T04:50:27.079033+0000","last_fullsized":"2024-09-16T04:50:27.079033+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":5292,"log_dups_size":0,"ondisk_log_size":5292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21643264,"num_objects":5284,"num_object_clones":0,"num_object_copies":10568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2362,"num_write_kb":9448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'5319","reported_seq":5542,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:26.760450+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:26.760450+0000","last_peered":"2024-09-16T04:50:26.760450+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:26.760450+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":5319,"log_dups_size":0,"ondisk_log_size":5319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21651456,"num_objects":5286,"num_object_clones":0,"num_object_copies":10572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2070,"num_write_kb":8280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":1965}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'5432","reported_seq":5555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.194334+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:27.194334+0000","last_peered":"2024-09-16T04:50:27.194334+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:27.194334+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":5432,"log_dups_size":0,"ondisk_log_size":5432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22216704,"num_objects":5424,"num_object_clones":0,"num_object_copies":10848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2087,"num_write_kb":8348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1959},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'5297","reported_seq":5534,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:26.709004+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:26.709004+0000","last_peered":"2024-09-16T04:50:26.709004+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:26.709004+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":5297,"log_dups_size":0,"ondisk_log_size":5297,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21570770,"num_objects":5268,"num_object_clones":0,"num_object_copies":10536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5269,"num_write_kb":21067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":1972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'5337","reported_seq":7607,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:26.768053+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:26.768053+0000","last_peered":"2024-09-16T04:50:26.768053+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:26.768053+0000","last_undegraded":"2024-09-16T04:50:26.768053+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":5337,"log_dups_size":0,"ondisk_log_size":5337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21835776,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2285,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2060,"num_write_kb":8240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1059,"num_bytes_recovered":4337664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2285},{"shards":"2","objects":3046}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'5330","reported_seq":5422,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:26.713263+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:26.713263+0000","last_peered":"2024-09-16T04:50:26.713263+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:26.713263+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21824105,"num_objects":5329,"num_object_clones":0,"num_object_copies":10658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5329,"num_write_kb":21313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":1922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'5416","reported_seq":5514,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:26.763851+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:26.763851+0000","last_peered":"2024-09-16T04:50:26.763851+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:26.763851+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":5416,"log_dups_size":0,"ondisk_log_size":5416,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22153019,"num_objects":5411,"num_object_clones":0,"num_object_copies":10822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5413,"num_write_kb":21636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":1897}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'5337","reported_seq":5463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.185990+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:27.185990+0000","last_peered":"2024-09-16T04:50:27.185990+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:27.185990+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":5337,"log_dups_size":0,"ondisk_log_size":5337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21843968,"num_objects":5333,"num_object_clones":0,"num_object_copies":10666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2091,"num_write_kb":8364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2008},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'5497","reported_seq":5632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.202466+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:27.202466+0000","last_peered":"2024-09-16T04:50:27.202466+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:27.202466+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":5497,"log_dups_size":0,"ondisk_log_size":5497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22467177,"num_objects":5486,"num_object_clones":0,"num_object_copies":10972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2266,"num_write_kb":9064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1962},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":350068959,"num_objects":85472,"num_object_clones":0,"num_object_copies":170944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2285,"num_objects_unfound":0,"num_objects_dirty":85472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50706,"num_write_kb":202796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1068,"num_bytes_recovered":4349952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":710402048,"data_stored":710315073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85628,"ondisk_log_size":85628,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461244,"kb_used_data":294892,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93910596,"statfs":{"total":96636764160,"available":96164450304,"internally_reserved":0,"allocated":301969408,"data_stored":301710646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":170323625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593740,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":229564,"kb_used_data":103468,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94142276,"statfs":{"total":96636764160,"available":96401690624,"internally_reserved":0,"allocated":105951232,"data_stored":105615932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":129101486},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457548,"kb_used_data":295420,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93914292,"statfs":{"total":96636764160,"available":96168235008,"internally_reserved":0,"allocated":302510080,"data_stored":302251318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":302432256,"data_stored":302428777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":106180608,"data_stored":106100591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":301789184,"data_stored":301785705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:30.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:50:30.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'5293", 'reported_seq': 5367, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:27.218014+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:50:27.218014+0000', 'last_peered': '2024-09-16T04:50:27.218014+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:50:27.218014+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 5293, 'log_dups_size': 0, 'ondisk_log_size': 5293, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21680128, 'num_objects': 5293, 'num_object_clones': 0, 'num_object_copies': 10586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5293, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5293, 'num_write_kb': 21172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1952}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:30.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:50:30.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'5262", 'reported_seq': 5395, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:26.674240+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:50:26.674240+0000', 'last_peered': '2024-09-16T04:50:26.674240+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:50:26.674240+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 5262, 'log_dups_size': 0, 'ondisk_log_size': 5262, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21536768, 'num_objects': 5258, 'num_object_clones': 0, 'num_object_copies': 10516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5258, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2001, 'num_write_kb': 8004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 1939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:30.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:50:30.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'5438", 'reported_seq': 5566, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:27.150266+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:50:27.150266+0000', 'last_peered': '2024-09-16T04:50:27.150266+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:50:27.150266+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 5438, 'log_dups_size': 0, 'ondisk_log_size': 5438, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22253568, 'num_objects': 5433, 'num_object_clones': 0, 'num_object_copies': 10866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5433, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2431, 'num_write_kb': 9724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1977}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:30.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:50:30.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'5306", 'reported_seq': 5449, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:27.186097+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:50:27.186097+0000', 'last_peered': '2024-09-16T04:50:27.186097+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:50:27.186097+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 5306, 'log_dups_size': 0, 'ondisk_log_size': 5306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21708800, 'num_objects': 5300, 'num_object_clones': 0, 'num_object_copies': 10600, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5300, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1961, 'num_write_kb': 7844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1901}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:30.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:50:30.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'5319", 'reported_seq': 5542, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:26.760450+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:50:26.760450+0000', 'last_peered': '2024-09-16T04:50:26.760450+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:50:26.760450+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 5319, 'log_dups_size': 0, 'ondisk_log_size': 5319, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21651456, 'num_objects': 5286, 'num_object_clones': 0, 'num_object_copies': 10572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5286, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2070, 'num_write_kb': 8280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 1965}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:30.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:50:30.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'5432", 'reported_seq': 5555, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:27.194334+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:50:27.194334+0000', 'last_peered': '2024-09-16T04:50:27.194334+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:50:27.194334+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 5432, 'log_dups_size': 0, 'ondisk_log_size': 5432, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22216704, 'num_objects': 5424, 'num_object_clones': 0, 'num_object_copies': 10848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5424, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2087, 'num_write_kb': 8348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1959}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:30.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:50:30.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'5297", 'reported_seq': 5534, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:26.709004+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:50:26.709004+0000', 'last_peered': '2024-09-16T04:50:26.709004+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:50:26.709004+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 5297, 'log_dups_size': 0, 'ondisk_log_size': 5297, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21570770, 'num_objects': 5268, 'num_object_clones': 0, 'num_object_copies': 10536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5268, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5269, 'num_write_kb': 21067, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 1972}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:30.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:50:30.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'5337", 'reported_seq': 7607, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:50:26.768053+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:50:26.768053+0000', 'last_peered': '2024-09-16T04:50:26.768053+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:50:26.768053+0000', 'last_undegraded': '2024-09-16T04:50:26.768053+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 5337, 'log_dups_size': 0, 'ondisk_log_size': 5337, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21835776, 'num_objects': 5331, 'num_object_clones': 0, 'num_object_copies': 10662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2285, 'num_objects_unfound': 0, 'num_objects_dirty': 5331, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2060, 'num_write_kb': 8240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1059, 'num_bytes_recovered': 4337664, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2285}, {'shards': '2', 'objects': 3046}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:30.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:50:30.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'5330", 'reported_seq': 5422, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:26.713263+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:50:26.713263+0000', 'last_peered': '2024-09-16T04:50:26.713263+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:50:26.713263+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 5330, 'log_dups_size': 0, 'ondisk_log_size': 5330, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21824105, 'num_objects': 5329, 'num_object_clones': 0, 'num_object_copies': 10658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5329, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5329, 'num_write_kb': 21313, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 1922}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:30.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:50:30.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'5416", 'reported_seq': 5514, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:26.763851+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:50:26.763851+0000', 'last_peered': '2024-09-16T04:50:26.763851+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:50:26.763851+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 5416, 'log_dups_size': 0, 'ondisk_log_size': 5416, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22153019, 'num_objects': 5411, 'num_object_clones': 0, 'num_object_copies': 10822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5411, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5413, 'num_write_kb': 21636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 1897}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:30.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:50:30.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'5337", 'reported_seq': 5463, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:27.185990+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:50:27.185990+0000', 'last_peered': '2024-09-16T04:50:27.185990+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:50:27.185990+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 5337, 'log_dups_size': 0, 'ondisk_log_size': 5337, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21843968, 'num_objects': 5333, 'num_object_clones': 0, 'num_object_copies': 10666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5333, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2091, 'num_write_kb': 8364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2008}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:30.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:50:30.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'5497", 'reported_seq': 5632, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:27.202466+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:50:27.202466+0000', 'last_peered': '2024-09-16T04:50:27.202466+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:50:27.202466+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 5497, 'log_dups_size': 0, 'ondisk_log_size': 5497, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22467177, 'num_objects': 5486, 'num_object_clones': 0, 'num_object_copies': 10972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5486, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2266, 'num_write_kb': 9064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1962}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:30.842 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:50:31.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 94 19 87919 87900 3.6519 2.20703 0.0597296 0.0213845 2024-09-16T04:50:32.004 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:32.004 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/170948 objects degraded (21.870%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":179,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":85474,"data_bytes":350068978,"bytes_used":1175916544,"bytes_avail":288734375936,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":170948,"degraded_ratio":0.21870393335985208,"misplaced_objects":2285,"misplaced_total":170948,"misplaced_ratio":0.013366637807988392,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":78830,"recovering_keys_per_sec":0,"num_objects_recovered":231,"num_bytes_recovered":946176,"num_keys_recovered":0,"write_bytes_sec":2057790,"read_op_per_sec":0,"write_op_per_sec":502},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-16T04:49:42.278701+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T04:50:32.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:50:32.046 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:32.068 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 95 20 88509 88489 3.63767 2.30078 0.057151 0.0214694 2024-09-16T04:50:33.057 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:33.057 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:33.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 96 20 89104 89084 3.62384 2.32422 0.0208463 0.0215504 2024-09-16T04:50:33.111 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-16T04:50:32.295082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":356167922,"num_objects":86963,"num_object_clones":0,"num_object_copies":173926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2154,"num_objects_unfound":0,"num_objects_dirty":86963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52197,"num_write_kb":208754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1200,"num_bytes_recovered":4886547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87120,"ondisk_log_size":87120,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1147652,"kb_used_data":701392,"kb_used_omap":12,"kb_used_meta":446195,"kb_avail":281967868,"statfs":{"total":289910292480,"available":288735096832,"internally_reserved":0,"allocated":718225408,"data_stored":717372584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":456903681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,5,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30797824,"num_objects":7519,"num_object_clones":0,"num_object_copies":15038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-362,"num_objects_unfound":0,"num_objects_dirty":7519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7519,"num_write_kb":30076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":1482752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002482"},"pg_stats":[{"pgid":"2.f","version":"45'5293","reported_seq":5367,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.218014+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:27.218014+0000","last_peered":"2024-09-16T04:50:27.218014+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:27.218014+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5293,"log_dups_size":0,"ondisk_log_size":5293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21680128,"num_objects":5293,"num_object_clones":0,"num_object_copies":10586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5293,"num_write_kb":21172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1952},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'5577","reported_seq":5688,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:31.684540+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:31.684540+0000","last_peered":"2024-09-16T04:50:31.684540+0000","last_clean":"2024-09-16T04:50:31.684540+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:31.684540+0000","last_undegraded":"2024-09-16T04:50:31.684540+0000","last_fullsized":"2024-09-16T04:50:31.684540+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":5577,"log_dups_size":0,"ondisk_log_size":5577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22818816,"num_objects":5571,"num_object_clones":0,"num_object_copies":11142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2546,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'5449","reported_seq":5582,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.764009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:31.764009+0000","last_peered":"2024-09-16T04:50:31.764009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:31.764009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":5449,"log_dups_size":0,"ondisk_log_size":5449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22302720,"num_objects":5445,"num_object_clones":0,"num_object_copies":10890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2188,"num_write_kb":8752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'5520","reported_seq":5584,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:31.765103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:31.765103+0000","last_peered":"2024-09-16T04:50:31.765103+0000","last_clean":"2024-09-16T04:50:31.765103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:31.765103+0000","last_undegraded":"2024-09-16T04:50:31.765103+0000","last_fullsized":"2024-09-16T04:50:31.765103+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5520,"log_dups_size":0,"ondisk_log_size":5520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5520,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'5375","reported_seq":5589,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:27.219123+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:27.219123+0000","last_peered":"2024-09-16T04:50:27.219123+0000","last_clean":"2024-09-16T04:50:27.219123+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:27.219123+0000","last_undegraded":"2024-09-16T04:50:27.219123+0000","last_fullsized":"2024-09-16T04:50:27.219123+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":5375,"log_dups_size":0,"ondisk_log_size":5375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21893120,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2407,"num_write_kb":9628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'5438","reported_seq":5566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.150266+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:27.150266+0000","last_peered":"2024-09-16T04:50:27.150266+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:27.150266+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":5438,"log_dups_size":0,"ondisk_log_size":5438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22253568,"num_objects":5433,"num_object_clones":0,"num_object_copies":10866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2431,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1977},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'5306","reported_seq":5449,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.186097+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:27.186097+0000","last_peered":"2024-09-16T04:50:27.186097+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:27.186097+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":5306,"log_dups_size":0,"ondisk_log_size":5306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21708800,"num_objects":5300,"num_object_clones":0,"num_object_copies":10600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1961,"num_write_kb":7844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1901},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'5292","reported_seq":5503,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:27.079033+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:27.079033+0000","last_peered":"2024-09-16T04:50:27.079033+0000","last_clean":"2024-09-16T04:50:27.079033+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:27.079033+0000","last_undegraded":"2024-09-16T04:50:27.079033+0000","last_fullsized":"2024-09-16T04:50:27.079033+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":5292,"log_dups_size":0,"ondisk_log_size":5292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21643264,"num_objects":5284,"num_object_clones":0,"num_object_copies":10568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2362,"num_write_kb":9448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'5497","reported_seq":5720,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.760195+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:31.760195+0000","last_peered":"2024-09-16T04:50:31.760195+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:31.760195+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":5497,"log_dups_size":0,"ondisk_log_size":5497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22380544,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2248,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":2143}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'5432","reported_seq":5555,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.194334+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:27.194334+0000","last_peered":"2024-09-16T04:50:27.194334+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:27.194334+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":5432,"log_dups_size":0,"ondisk_log_size":5432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22216704,"num_objects":5424,"num_object_clones":0,"num_object_copies":10848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2087,"num_write_kb":8348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1959},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'5475","reported_seq":5712,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.760173+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:31.760173+0000","last_peered":"2024-09-16T04:50:31.760173+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:31.760173+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22299858,"num_objects":5446,"num_object_clones":0,"num_object_copies":10892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5447,"num_write_kb":21779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":2150}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'5526","reported_seq":8059,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:31.752265+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:31.752265+0000","last_peered":"2024-09-16T04:50:31.752265+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:31.752265+0000","last_undegraded":"2024-09-16T04:50:31.752265+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":5526,"log_dups_size":0,"ondisk_log_size":5526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2154,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2249,"num_write_kb":8996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1190,"num_bytes_recovered":4874240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2154},{"shards":"2","objects":3366}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'5505","reported_seq":5598,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.744397+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:31.744397+0000","last_peered":"2024-09-16T04:50:31.744397+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:31.744397+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5505,"log_dups_size":0,"ondisk_log_size":5505,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22540905,"num_objects":5504,"num_object_clones":0,"num_object_copies":11008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5504,"num_write_kb":22013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":2097}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'5598","reported_seq":5696,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.740427+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:31.740427+0000","last_peered":"2024-09-16T04:50:31.740427+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:31.740427+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":5598,"log_dups_size":0,"ondisk_log_size":5598,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22898491,"num_objects":5593,"num_object_clones":0,"num_object_copies":11186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5595,"num_write_kb":22364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":2079}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'5337","reported_seq":5463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.185990+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:27.185990+0000","last_peered":"2024-09-16T04:50:27.185990+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:27.185990+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":5337,"log_dups_size":0,"ondisk_log_size":5337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21843968,"num_objects":5333,"num_object_clones":0,"num_object_copies":10666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2091,"num_write_kb":8364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2008},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'5497","reported_seq":5632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:27.202466+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:27.202466+0000","last_peered":"2024-09-16T04:50:27.202466+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:27.202466+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":5497,"log_dups_size":0,"ondisk_log_size":5497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22467177,"num_objects":5486,"num_object_clones":0,"num_object_copies":10972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2266,"num_write_kb":9064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1962},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":356167903,"num_objects":86961,"num_object_clones":0,"num_object_copies":173922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2154,"num_objects_unfound":0,"num_objects_dirty":86961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52195,"num_write_kb":208752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1199,"num_bytes_recovered":4886528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":718016512,"data_stored":717929537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87117,"ondisk_log_size":87117,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460540,"kb_used_data":302504,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93911300,"statfs":{"total":96636764160,"available":96165171200,"internally_reserved":0,"allocated":309764096,"data_stored":309505334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":161803945},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593740,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":229564,"kb_used_data":103468,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94142276,"statfs":{"total":96636764160,"available":96401690624,"internally_reserved":0,"allocated":105951232,"data_stored":105615932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":129101486},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457548,"kb_used_data":295420,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93914292,"statfs":{"total":96636764160,"available":96168235008,"internally_reserved":0,"allocated":302510080,"data_stored":302251318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":302432256,"data_stored":302428777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":106180608,"data_stored":106100591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":309403648,"data_stored":309400169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:34.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 97 20 89693 89673 3.6102 2.30078 0.0607223 0.0216315 2024-09-16T04:50:35.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 98 20 90284 90264 3.59692 2.30859 0.0887004 0.02171 2024-09-16T04:50:36.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 99 20 90888 90868 3.58442 2.35938 0.0262211 0.021787 2024-09-16T04:50:36.113 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:37.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:2024-09-16T04:50:37.071812+0000 min lat: 0.00206892 max lat: 2.69414 avg lat: 0.0218729 2024-09-16T04:50:37.073 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T04:50:37.073 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 100 20 91455 91435 3.57072 2.21484 0.0400484 0.0218729 2024-09-16T04:50:37.128 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:37.129 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:37.205 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-09-16T04:50:36.296080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":362381554,"num_objects":88480,"num_object_clones":0,"num_object_copies":176960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2154,"num_objects_unfound":0,"num_objects_dirty":88480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53714,"num_write_kb":214822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1200,"num_bytes_recovered":4886547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88637,"ondisk_log_size":88637,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1164380,"kb_used_data":718120,"kb_used_omap":12,"kb_used_meta":446195,"kb_avail":281951140,"statfs":{"total":289910292480,"available":288717967360,"internally_reserved":0,"allocated":735354880,"data_stored":734502056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":456903681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,7,11,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24547328,"num_objects":5993,"num_object_clones":0,"num_object_copies":11986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-247,"num_objects_unfound":0,"num_objects_dirty":5993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5993,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":1011712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002608"},"pg_stats":[{"pgid":"2.f","version":"45'5480","reported_seq":5554,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:32.256904+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:32.256904+0000","last_peered":"2024-09-16T04:50:32.256904+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:32.256904+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5480,"log_dups_size":0,"ondisk_log_size":5480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22446080,"num_objects":5480,"num_object_clones":0,"num_object_copies":10960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5480,"num_write_kb":21920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2139},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'5577","reported_seq":5688,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:31.684540+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:31.684540+0000","last_peered":"2024-09-16T04:50:31.684540+0000","last_clean":"2024-09-16T04:50:31.684540+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:31.684540+0000","last_undegraded":"2024-09-16T04:50:31.684540+0000","last_fullsized":"2024-09-16T04:50:31.684540+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":5577,"log_dups_size":0,"ondisk_log_size":5577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22818816,"num_objects":5571,"num_object_clones":0,"num_object_copies":11142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2546,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'5449","reported_seq":5582,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.764009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:31.764009+0000","last_peered":"2024-09-16T04:50:31.764009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:31.764009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":5449,"log_dups_size":0,"ondisk_log_size":5449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22302720,"num_objects":5445,"num_object_clones":0,"num_object_copies":10890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2188,"num_write_kb":8752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'5520","reported_seq":5584,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:31.765103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:31.765103+0000","last_peered":"2024-09-16T04:50:31.765103+0000","last_clean":"2024-09-16T04:50:31.765103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:31.765103+0000","last_undegraded":"2024-09-16T04:50:31.765103+0000","last_fullsized":"2024-09-16T04:50:31.765103+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5520,"log_dups_size":0,"ondisk_log_size":5520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5520,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'5567","reported_seq":5780,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:32.244515+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:32.244515+0000","last_peered":"2024-09-16T04:50:32.244515+0000","last_clean":"2024-09-16T04:50:32.244515+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:32.244515+0000","last_undegraded":"2024-09-16T04:50:32.244515+0000","last_fullsized":"2024-09-16T04:50:32.244515+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":5567,"log_dups_size":0,"ondisk_log_size":5567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22679552,"num_objects":5537,"num_object_clones":0,"num_object_copies":11074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2599,"num_write_kb":10396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'5625","reported_seq":5753,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:32.210713+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:32.210713+0000","last_peered":"2024-09-16T04:50:32.210713+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:32.210713+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":5625,"log_dups_size":0,"ondisk_log_size":5625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23019520,"num_objects":5620,"num_object_clones":0,"num_object_copies":11240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2618,"num_write_kb":10472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2164},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'5494","reported_seq":5637,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:32.175995+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:32.175995+0000","last_peered":"2024-09-16T04:50:32.175995+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:32.175995+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":5494,"log_dups_size":0,"ondisk_log_size":5494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22478848,"num_objects":5488,"num_object_clones":0,"num_object_copies":10976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2149,"num_write_kb":8596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2089},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'5473","reported_seq":5684,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:32.123872+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:32.123872+0000","last_peered":"2024-09-16T04:50:32.123872+0000","last_clean":"2024-09-16T04:50:32.123872+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:32.123872+0000","last_undegraded":"2024-09-16T04:50:32.123872+0000","last_fullsized":"2024-09-16T04:50:32.123872+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":5473,"log_dups_size":0,"ondisk_log_size":5473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22384640,"num_objects":5465,"num_object_clones":0,"num_object_copies":10930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2543,"num_write_kb":10172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'5497","reported_seq":5720,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.760195+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:31.760195+0000","last_peered":"2024-09-16T04:50:31.760195+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:31.760195+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":5497,"log_dups_size":0,"ondisk_log_size":5497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22380544,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2248,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":2143}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'5632","reported_seq":5757,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:32.205693+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:32.205693+0000","last_peered":"2024-09-16T04:50:32.205693+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:32.205693+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":5632,"log_dups_size":0,"ondisk_log_size":5632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23035904,"num_objects":5624,"num_object_clones":0,"num_object_copies":11248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2287,"num_write_kb":9148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2159},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'5475","reported_seq":5712,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.760173+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:31.760173+0000","last_peered":"2024-09-16T04:50:31.760173+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:31.760173+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22299858,"num_objects":5446,"num_object_clones":0,"num_object_copies":10892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5447,"num_write_kb":21779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":2150}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'5526","reported_seq":8059,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:31.752265+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:31.752265+0000","last_peered":"2024-09-16T04:50:31.752265+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:31.752265+0000","last_undegraded":"2024-09-16T04:50:31.752265+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":5526,"log_dups_size":0,"ondisk_log_size":5526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2154,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2249,"num_write_kb":8996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1190,"num_bytes_recovered":4874240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2154},{"shards":"2","objects":3366}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'5505","reported_seq":5598,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.744397+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:31.744397+0000","last_peered":"2024-09-16T04:50:31.744397+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:31.744397+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5505,"log_dups_size":0,"ondisk_log_size":5505,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22540905,"num_objects":5504,"num_object_clones":0,"num_object_copies":11008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5504,"num_write_kb":22013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":2097}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'5598","reported_seq":5696,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.740427+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:31.740427+0000","last_peered":"2024-09-16T04:50:31.740427+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:31.740427+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":5598,"log_dups_size":0,"ondisk_log_size":5598,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22898491,"num_objects":5593,"num_object_clones":0,"num_object_copies":11186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5595,"num_write_kb":22364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":2079}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'5529","reported_seq":5655,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:32.264328+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:32.264328+0000","last_peered":"2024-09-16T04:50:32.264328+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:32.264328+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":5529,"log_dups_size":0,"ondisk_log_size":5529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22630400,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2283,"num_write_kb":9132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2200},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'5687","reported_seq":5822,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:32.188866+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:32.188866+0000","last_peered":"2024-09-16T04:50:32.188866+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:32.188866+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":5687,"log_dups_size":0,"ondisk_log_size":5687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23245417,"num_objects":5676,"num_object_clones":0,"num_object_copies":11352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2456,"num_write_kb":9824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2152},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":362381535,"num_objects":88478,"num_object_clones":0,"num_object_copies":176956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2154,"num_objects_unfound":0,"num_objects_dirty":88478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53712,"num_write_kb":214820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1199,"num_bytes_recovered":4886528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":735510528,"data_stored":735423553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88634,"ondisk_log_size":88634,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460540,"kb_used_data":302504,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93911300,"statfs":{"total":96636764160,"available":96165171200,"internally_reserved":0,"allocated":309764096,"data_stored":309505334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":161803945},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593741,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246904,"kb_used_data":112488,"kb_used_omap":4,"kb_used_meta":134395,"kb_avail":94124936,"statfs":{"total":96636764160,"available":96383934464,"internally_reserved":0,"allocated":115187712,"data_stored":114852412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":137621166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456936,"kb_used_data":303128,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93914904,"statfs":{"total":96636764160,"available":96168861696,"internally_reserved":0,"allocated":310403072,"data_stored":310144310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157478570},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":310337536,"data_stored":310334057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115769344,"data_stored":115689327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":309403648,"data_stored":309400169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:37.207 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:38.051 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:38.054 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:38.072 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 101 20 92023 92003 3.55734 2.21875 0.0255392 0.021953 2024-09-16T04:50:38.115 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-09-16T04:50:36.296080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":362381554,"num_objects":88480,"num_object_clones":0,"num_object_copies":176960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2154,"num_objects_unfound":0,"num_objects_dirty":88480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53714,"num_write_kb":214822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1200,"num_bytes_recovered":4886547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88637,"ondisk_log_size":88637,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1164380,"kb_used_data":718120,"kb_used_omap":12,"kb_used_meta":446195,"kb_avail":281951140,"statfs":{"total":289910292480,"available":288717967360,"internally_reserved":0,"allocated":735354880,"data_stored":734502056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":456903681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,7,11,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24547328,"num_objects":5993,"num_object_clones":0,"num_object_copies":11986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-247,"num_objects_unfound":0,"num_objects_dirty":5993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5993,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":1011712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002608"},"pg_stats":[{"pgid":"2.f","version":"45'5480","reported_seq":5554,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:32.256904+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:32.256904+0000","last_peered":"2024-09-16T04:50:32.256904+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:32.256904+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5480,"log_dups_size":0,"ondisk_log_size":5480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22446080,"num_objects":5480,"num_object_clones":0,"num_object_copies":10960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5480,"num_write_kb":21920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2139},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'5577","reported_seq":5688,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:31.684540+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:31.684540+0000","last_peered":"2024-09-16T04:50:31.684540+0000","last_clean":"2024-09-16T04:50:31.684540+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:31.684540+0000","last_undegraded":"2024-09-16T04:50:31.684540+0000","last_fullsized":"2024-09-16T04:50:31.684540+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":5577,"log_dups_size":0,"ondisk_log_size":5577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22818816,"num_objects":5571,"num_object_clones":0,"num_object_copies":11142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2546,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'5449","reported_seq":5582,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.764009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:31.764009+0000","last_peered":"2024-09-16T04:50:31.764009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:31.764009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":5449,"log_dups_size":0,"ondisk_log_size":5449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22302720,"num_objects":5445,"num_object_clones":0,"num_object_copies":10890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2188,"num_write_kb":8752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'5520","reported_seq":5584,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:31.765103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:31.765103+0000","last_peered":"2024-09-16T04:50:31.765103+0000","last_clean":"2024-09-16T04:50:31.765103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:31.765103+0000","last_undegraded":"2024-09-16T04:50:31.765103+0000","last_fullsized":"2024-09-16T04:50:31.765103+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5520,"log_dups_size":0,"ondisk_log_size":5520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5520,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'5567","reported_seq":5780,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:32.244515+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:32.244515+0000","last_peered":"2024-09-16T04:50:32.244515+0000","last_clean":"2024-09-16T04:50:32.244515+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:32.244515+0000","last_undegraded":"2024-09-16T04:50:32.244515+0000","last_fullsized":"2024-09-16T04:50:32.244515+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":5567,"log_dups_size":0,"ondisk_log_size":5567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22679552,"num_objects":5537,"num_object_clones":0,"num_object_copies":11074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2599,"num_write_kb":10396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'5625","reported_seq":5753,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:32.210713+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:32.210713+0000","last_peered":"2024-09-16T04:50:32.210713+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:32.210713+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":5625,"log_dups_size":0,"ondisk_log_size":5625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23019520,"num_objects":5620,"num_object_clones":0,"num_object_copies":11240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2618,"num_write_kb":10472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2164},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'5494","reported_seq":5637,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:32.175995+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:32.175995+0000","last_peered":"2024-09-16T04:50:32.175995+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:32.175995+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":5494,"log_dups_size":0,"ondisk_log_size":5494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22478848,"num_objects":5488,"num_object_clones":0,"num_object_copies":10976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2149,"num_write_kb":8596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2089},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'5473","reported_seq":5684,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:32.123872+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:32.123872+0000","last_peered":"2024-09-16T04:50:32.123872+0000","last_clean":"2024-09-16T04:50:32.123872+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:32.123872+0000","last_undegraded":"2024-09-16T04:50:32.123872+0000","last_fullsized":"2024-09-16T04:50:32.123872+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":5473,"log_dups_size":0,"ondisk_log_size":5473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22384640,"num_objects":5465,"num_object_clones":0,"num_object_copies":10930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2543,"num_write_kb":10172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'5497","reported_seq":5720,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.760195+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:31.760195+0000","last_peered":"2024-09-16T04:50:31.760195+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:31.760195+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":5497,"log_dups_size":0,"ondisk_log_size":5497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22380544,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2248,"num_write_kb":8992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":2143}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'5632","reported_seq":5757,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:32.205693+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:32.205693+0000","last_peered":"2024-09-16T04:50:32.205693+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:32.205693+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":5632,"log_dups_size":0,"ondisk_log_size":5632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23035904,"num_objects":5624,"num_object_clones":0,"num_object_copies":11248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2287,"num_write_kb":9148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2159},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'5475","reported_seq":5712,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.760173+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:31.760173+0000","last_peered":"2024-09-16T04:50:31.760173+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:31.760173+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":5475,"log_dups_size":0,"ondisk_log_size":5475,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22299858,"num_objects":5446,"num_object_clones":0,"num_object_copies":10892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5447,"num_write_kb":21779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":2150}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'5526","reported_seq":8059,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:31.752265+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:31.752265+0000","last_peered":"2024-09-16T04:50:31.752265+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:31.752265+0000","last_undegraded":"2024-09-16T04:50:31.752265+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":5526,"log_dups_size":0,"ondisk_log_size":5526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22609920,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2154,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2249,"num_write_kb":8996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1190,"num_bytes_recovered":4874240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2154},{"shards":"2","objects":3366}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'5505","reported_seq":5598,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.744397+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:31.744397+0000","last_peered":"2024-09-16T04:50:31.744397+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:31.744397+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5505,"log_dups_size":0,"ondisk_log_size":5505,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22540905,"num_objects":5504,"num_object_clones":0,"num_object_copies":11008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5504,"num_write_kb":22013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":2097}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'5598","reported_seq":5696,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:31.740427+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:31.740427+0000","last_peered":"2024-09-16T04:50:31.740427+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:31.740427+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":5598,"log_dups_size":0,"ondisk_log_size":5598,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22898491,"num_objects":5593,"num_object_clones":0,"num_object_copies":11186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5595,"num_write_kb":22364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":2079}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'5529","reported_seq":5655,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:32.264328+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:32.264328+0000","last_peered":"2024-09-16T04:50:32.264328+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:32.264328+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":5529,"log_dups_size":0,"ondisk_log_size":5529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22630400,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2283,"num_write_kb":9132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2200},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'5687","reported_seq":5822,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:32.188866+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:32.188866+0000","last_peered":"2024-09-16T04:50:32.188866+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:32.188866+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":5687,"log_dups_size":0,"ondisk_log_size":5687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23245417,"num_objects":5676,"num_object_clones":0,"num_object_copies":11352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2456,"num_write_kb":9824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2152},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":362381535,"num_objects":88478,"num_object_clones":0,"num_object_copies":176956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2154,"num_objects_unfound":0,"num_objects_dirty":88478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53712,"num_write_kb":214820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1199,"num_bytes_recovered":4886528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":735510528,"data_stored":735423553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88634,"ondisk_log_size":88634,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460540,"kb_used_data":302504,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93911300,"statfs":{"total":96636764160,"available":96165171200,"internally_reserved":0,"allocated":309764096,"data_stored":309505334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":161803945},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593741,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246904,"kb_used_data":112488,"kb_used_omap":4,"kb_used_meta":134395,"kb_avail":94124936,"statfs":{"total":96636764160,"available":96383934464,"internally_reserved":0,"allocated":115187712,"data_stored":114852412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":137621166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456936,"kb_used_data":303128,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93914904,"statfs":{"total":96636764160,"available":96168861696,"internally_reserved":0,"allocated":310403072,"data_stored":310144310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":157478570},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":310337536,"data_stored":310334057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115769344,"data_stored":115689327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":309403648,"data_stored":309400169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:38.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:50:38.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'5480", 'reported_seq': 5554, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:32.256904+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:50:32.256904+0000', 'last_peered': '2024-09-16T04:50:32.256904+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:50:32.256904+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 5480, 'log_dups_size': 0, 'ondisk_log_size': 5480, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22446080, 'num_objects': 5480, 'num_object_clones': 0, 'num_object_copies': 10960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5480, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5480, 'num_write_kb': 21920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2139}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:38.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:50:38.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'5449", 'reported_seq': 5582, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:31.764009+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:50:31.764009+0000', 'last_peered': '2024-09-16T04:50:31.764009+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:50:31.764009+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 5449, 'log_dups_size': 0, 'ondisk_log_size': 5449, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22302720, 'num_objects': 5445, 'num_object_clones': 0, 'num_object_copies': 10890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5445, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2188, 'num_write_kb': 8752, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 2126}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:38.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:50:38.117 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'5625", 'reported_seq': 5753, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:32.210713+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:50:32.210713+0000', 'last_peered': '2024-09-16T04:50:32.210713+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:50:32.210713+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 5625, 'log_dups_size': 0, 'ondisk_log_size': 5625, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23019520, 'num_objects': 5620, 'num_object_clones': 0, 'num_object_copies': 11240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5620, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2618, 'num_write_kb': 10472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2164}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:38.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:50:38.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'5494", 'reported_seq': 5637, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:32.175995+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:50:32.175995+0000', 'last_peered': '2024-09-16T04:50:32.175995+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:50:32.175995+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 5494, 'log_dups_size': 0, 'ondisk_log_size': 5494, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22478848, 'num_objects': 5488, 'num_object_clones': 0, 'num_object_copies': 10976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5488, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2149, 'num_write_kb': 8596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2089}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:38.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:50:38.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'5497", 'reported_seq': 5720, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:31.760195+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:50:31.760195+0000', 'last_peered': '2024-09-16T04:50:31.760195+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:50:31.760195+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 5497, 'log_dups_size': 0, 'ondisk_log_size': 5497, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22380544, 'num_objects': 5464, 'num_object_clones': 0, 'num_object_copies': 10928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2248, 'num_write_kb': 8992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 2143}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:38.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:50:38.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'5632", 'reported_seq': 5757, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:32.205693+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:50:32.205693+0000', 'last_peered': '2024-09-16T04:50:32.205693+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:50:32.205693+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 5632, 'log_dups_size': 0, 'ondisk_log_size': 5632, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23035904, 'num_objects': 5624, 'num_object_clones': 0, 'num_object_copies': 11248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2287, 'num_write_kb': 9148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2159}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:38.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:50:38.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'5475", 'reported_seq': 5712, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:31.760173+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:50:31.760173+0000', 'last_peered': '2024-09-16T04:50:31.760173+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:50:31.760173+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 5475, 'log_dups_size': 0, 'ondisk_log_size': 5475, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22299858, 'num_objects': 5446, 'num_object_clones': 0, 'num_object_copies': 10892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5446, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5447, 'num_write_kb': 21779, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 2150}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:38.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:50:38.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'5526", 'reported_seq': 8059, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:50:31.752265+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:50:31.752265+0000', 'last_peered': '2024-09-16T04:50:31.752265+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:50:31.752265+0000', 'last_undegraded': '2024-09-16T04:50:31.752265+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 5526, 'log_dups_size': 0, 'ondisk_log_size': 5526, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22609920, 'num_objects': 5520, 'num_object_clones': 0, 'num_object_copies': 11040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2154, 'num_objects_unfound': 0, 'num_objects_dirty': 5520, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2249, 'num_write_kb': 8996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1190, 'num_bytes_recovered': 4874240, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2154}, {'shards': '2', 'objects': 3366}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:38.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:50:38.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'5505", 'reported_seq': 5598, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:31.744397+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:50:31.744397+0000', 'last_peered': '2024-09-16T04:50:31.744397+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:50:31.744397+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 5505, 'log_dups_size': 0, 'ondisk_log_size': 5505, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22540905, 'num_objects': 5504, 'num_object_clones': 0, 'num_object_copies': 11008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5504, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5504, 'num_write_kb': 22013, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 2097}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:38.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:50:38.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'5598", 'reported_seq': 5696, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:31.740427+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:50:31.740427+0000', 'last_peered': '2024-09-16T04:50:31.740427+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:50:31.740427+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 5598, 'log_dups_size': 0, 'ondisk_log_size': 5598, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22898491, 'num_objects': 5593, 'num_object_clones': 0, 'num_object_copies': 11186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5593, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5595, 'num_write_kb': 22364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 2079}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:38.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:50:38.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'5529", 'reported_seq': 5655, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:32.264328+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:50:32.264328+0000', 'last_peered': '2024-09-16T04:50:32.264328+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:50:32.264328+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 5529, 'log_dups_size': 0, 'ondisk_log_size': 5529, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22630400, 'num_objects': 5525, 'num_object_clones': 0, 'num_object_copies': 11050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5525, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2283, 'num_write_kb': 9132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2200}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:38.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:50:38.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'5687", 'reported_seq': 5822, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:32.188866+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:50:32.188866+0000', 'last_peered': '2024-09-16T04:50:32.188866+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:50:32.188866+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 5687, 'log_dups_size': 0, 'ondisk_log_size': 5687, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23245417, 'num_objects': 5676, 'num_object_clones': 0, 'num_object_copies': 11352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5676, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2456, 'num_write_kb': 9824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2152}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:38.119 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:50:39.073 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 102 20 92592 92572 3.54425 2.22266 0.0995976 0.0220322 2024-09-16T04:50:39.290 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:39.290 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/182938 objects degraded (20.437%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":187,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":91469,"data_bytes":374624498,"bytes_used":1251483648,"bytes_avail":288658808832,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":182938,"degraded_ratio":0.20436978648503865,"misplaced_objects":2005,"misplaced_total":182938,"misplaced_ratio":0.010959997376160229,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":135138,"recovering_keys_per_sec":0,"num_objects_recovered":396,"num_bytes_recovered":1622016,"num_keys_recovered":0,"write_bytes_sec":3065183,"read_op_per_sec":0,"write_op_per_sec":748},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:50:39.391 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:50:39.392 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:40.073 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 103 20 93160 93140 3.53138 2.21875 0.026443 0.0221154 2024-09-16T04:50:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:40.285 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:40.360 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-16T04:50:38.296533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":374624498,"num_objects":91469,"num_object_clones":0,"num_object_copies":182938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2005,"num_objects_unfound":0,"num_objects_dirty":91469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56703,"num_write_kb":226778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1349,"num_bytes_recovered":5496851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91626,"ondisk_log_size":91626,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1222152,"kb_used_data":731480,"kb_used_omap":12,"kb_used_meta":490611,"kb_avail":281893368,"statfs":{"total":289910292480,"available":288658808832,"internally_reserved":0,"allocated":749035520,"data_stored":748182696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":502385665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,4,8,9,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":36790272,"num_objects":8982,"num_object_clones":0,"num_object_copies":17964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-396,"num_objects_unfound":0,"num_objects_dirty":8982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8982,"num_write_kb":35928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":1622016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002631"},"pg_stats":[{"pgid":"2.f","version":"45'5658","reported_seq":5732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:37.284110+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:37.284110+0000","last_peered":"2024-09-16T04:50:37.284110+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:37.284110+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5658,"log_dups_size":0,"ondisk_log_size":5658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23175168,"num_objects":5658,"num_object_clones":0,"num_object_copies":11316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5658,"num_write_kb":22632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2317},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'5750","reported_seq":5860,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:36.744045+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:36.744045+0000","last_peered":"2024-09-16T04:50:36.744045+0000","last_clean":"2024-09-16T04:50:36.744045+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:36.744045+0000","last_undegraded":"2024-09-16T04:50:36.744045+0000","last_fullsized":"2024-09-16T04:50:36.744045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":5750,"log_dups_size":0,"ondisk_log_size":5750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23527424,"num_objects":5744,"num_object_clones":0,"num_object_copies":11488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2719,"num_write_kb":10876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'5647","reported_seq":5779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:36.760614+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:36.760614+0000","last_peered":"2024-09-16T04:50:36.760614+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:36.760614+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":5647,"log_dups_size":0,"ondisk_log_size":5647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23113728,"num_objects":5643,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2386,"num_write_kb":9544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":2324}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'5722","reported_seq":5787,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:36.784999+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:36.784999+0000","last_peered":"2024-09-16T04:50:36.784999+0000","last_clean":"2024-09-16T04:50:36.784999+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:36.784999+0000","last_undegraded":"2024-09-16T04:50:36.784999+0000","last_fullsized":"2024-09-16T04:50:36.784999+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5722,"log_dups_size":0,"ondisk_log_size":5722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23437312,"num_objects":5722,"num_object_clones":0,"num_object_copies":11444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5722,"num_write_kb":22888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'5747","reported_seq":5957,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:37.285345+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:37.285345+0000","last_peered":"2024-09-16T04:50:37.285345+0000","last_clean":"2024-09-16T04:50:37.285345+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:37.285345+0000","last_undegraded":"2024-09-16T04:50:37.285345+0000","last_fullsized":"2024-09-16T04:50:37.285345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":5747,"log_dups_size":0,"ondisk_log_size":5747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23416832,"num_objects":5717,"num_object_clones":0,"num_object_copies":11434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2779,"num_write_kb":11116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'5813","reported_seq":5941,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:37.290263+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:37.290263+0000","last_peered":"2024-09-16T04:50:37.290263+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:37.290263+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":5813,"log_dups_size":0,"ondisk_log_size":5813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23789568,"num_objects":5808,"num_object_clones":0,"num_object_copies":11616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2806,"num_write_kb":11224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2352},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'5676","reported_seq":5819,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:37.287855+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:37.287855+0000","last_peered":"2024-09-16T04:50:37.287855+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:37.287855+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":5676,"log_dups_size":0,"ondisk_log_size":5676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23224320,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2331,"num_write_kb":9324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2271},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'5658","reported_seq":5868,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:37.241210+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:37.241210+0000","last_peered":"2024-09-16T04:50:37.241210+0000","last_clean":"2024-09-16T04:50:37.241210+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:37.241210+0000","last_undegraded":"2024-09-16T04:50:37.241210+0000","last_fullsized":"2024-09-16T04:50:37.241210+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":5658,"log_dups_size":0,"ondisk_log_size":5658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23142400,"num_objects":5650,"num_object_clones":0,"num_object_copies":11300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2728,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'5686","reported_seq":5909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:36.785977+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:36.785977+0000","last_peered":"2024-09-16T04:50:36.785977+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:36.785977+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":5686,"log_dups_size":0,"ondisk_log_size":5686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23154688,"num_objects":5653,"num_object_clones":0,"num_object_copies":11306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2437,"num_write_kb":9748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":2332}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'5826","reported_seq":5951,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:37.309883+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:37.309883+0000","last_peered":"2024-09-16T04:50:37.309883+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:37.309883+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":5826,"log_dups_size":0,"ondisk_log_size":5826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23830528,"num_objects":5818,"num_object_clones":0,"num_object_copies":11636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2481,"num_write_kb":9924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2353},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'5663","reported_seq":5900,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:36.742109+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:36.742109+0000","last_peered":"2024-09-16T04:50:36.742109+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:36.742109+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":5663,"log_dups_size":0,"ondisk_log_size":5663,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23069906,"num_objects":5634,"num_object_clones":0,"num_object_copies":11268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5635,"num_write_kb":22531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":2338}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'5701","reported_seq":8532,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:36.802716+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:36.802716+0000","last_peered":"2024-09-16T04:50:36.802716+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:36.802716+0000","last_undegraded":"2024-09-16T04:50:36.802716+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23326720,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2005,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2424,"num_write_kb":9696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1339,"num_bytes_recovered":5484544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2005},{"shards":"2","objects":3690}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'5691","reported_seq":5784,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:36.793845+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:36.793845+0000","last_peered":"2024-09-16T04:50:36.793845+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:36.793845+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5691,"log_dups_size":0,"ondisk_log_size":5691,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23302761,"num_objects":5690,"num_object_clones":0,"num_object_copies":11380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5690,"num_write_kb":22757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":2283}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'5794","reported_seq":5892,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:36.806637+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:36.806637+0000","last_peered":"2024-09-16T04:50:36.806637+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:36.806637+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":5794,"log_dups_size":0,"ondisk_log_size":5794,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23701307,"num_objects":5789,"num_object_clones":0,"num_object_copies":11578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5791,"num_write_kb":23148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":2275}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'5726","reported_seq":5852,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:37.326684+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:37.326684+0000","last_peered":"2024-09-16T04:50:37.326684+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:37.326684+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":5726,"log_dups_size":0,"ondisk_log_size":5726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23437312,"num_objects":5722,"num_object_clones":0,"num_object_copies":11444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2480,"num_write_kb":9920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2397},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'5865","reported_seq":6000,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:37.318443+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:37.318443+0000","last_peered":"2024-09-16T04:50:37.318443+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:37.318443+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":5865,"log_dups_size":0,"ondisk_log_size":5865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23974505,"num_objects":5854,"num_object_clones":0,"num_object_copies":11708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2634,"num_write_kb":10536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2330},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":374624479,"num_objects":91467,"num_object_clones":0,"num_object_copies":182934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":2005,"num_objects_unfound":0,"num_objects_dirty":91467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56701,"num_write_kb":226776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1348,"num_bytes_recovered":5496832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":750653440,"data_stored":750566465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91623,"ondisk_log_size":91623,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488772,"kb_used_data":308532,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93883068,"statfs":{"total":96636764160,"available":96136261632,"internally_reserved":0,"allocated":315936768,"data_stored":315678006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":184544937},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593741,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":246904,"kb_used_data":112488,"kb_used_omap":4,"kb_used_meta":134395,"kb_avail":94124936,"statfs":{"total":96636764160,"available":96383934464,"internally_reserved":0,"allocated":115187712,"data_stored":114852412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":137621166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486476,"kb_used_data":310460,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93885364,"statfs":{"total":96636764160,"available":96138612736,"internally_reserved":0,"allocated":317911040,"data_stored":317652278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":317841408,"data_stored":317837929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115769344,"data_stored":115689327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317042688,"data_stored":317039209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:41.076 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 104 20 93735 93715 3.51893 2.24609 0.0406269 0.0221927 2024-09-16T04:50:42.076 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 105 20 94339 94319 3.50788 2.35938 0.00666325 0.0222582 2024-09-16T04:50:43.076 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 106 20 94942 94922 3.49701 2.35547 0.0157586 0.0223338 2024-09-16T04:50:43.363 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:44.076 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 107 20 95511 95491 3.4851 2.22266 0.0276963 0.0224076 2024-09-16T04:50:44.274 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:44.276 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:44.327 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-16T04:50:42.297539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":380440818,"num_objects":92889,"num_object_clones":0,"num_object_copies":185778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1871,"num_objects_unfound":0,"num_objects_dirty":92889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58123,"num_write_kb":232458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1483,"num_bytes_recovered":6045715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93046,"ondisk_log_size":93046,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1239752,"kb_used_data":749080,"kb_used_omap":12,"kb_used_meta":490611,"kb_avail":281875768,"statfs":{"total":289910292480,"available":288640786432,"internally_reserved":0,"allocated":767057920,"data_stored":766205096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":502385665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,5,6,12,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30371840,"num_objects":7415,"num_object_clones":0,"num_object_copies":14830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-414,"num_objects_unfound":0,"num_objects_dirty":7415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7415,"num_write_kb":29660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":1695744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002786"},"pg_stats":[{"pgid":"2.f","version":"45'5658","reported_seq":5732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:37.284110+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:37.284110+0000","last_peered":"2024-09-16T04:50:37.284110+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:37.284110+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5658,"log_dups_size":0,"ondisk_log_size":5658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23175168,"num_objects":5658,"num_object_clones":0,"num_object_copies":11316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5658,"num_write_kb":22632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2317},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'5930","reported_seq":6041,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:41.820396+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:41.820396+0000","last_peered":"2024-09-16T04:50:41.820396+0000","last_clean":"2024-09-16T04:50:41.820396+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:41.820396+0000","last_undegraded":"2024-09-16T04:50:41.820396+0000","last_fullsized":"2024-09-16T04:50:41.820396+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":5930,"log_dups_size":0,"ondisk_log_size":5930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264704,"num_objects":5924,"num_object_clones":0,"num_object_copies":11848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2899,"num_write_kb":11596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'5823","reported_seq":5956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.750987+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:41.750987+0000","last_peered":"2024-09-16T04:50:41.750987+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:41.750987+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":5823,"log_dups_size":0,"ondisk_log_size":5823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23834624,"num_objects":5819,"num_object_clones":0,"num_object_copies":11638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2562,"num_write_kb":10248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":2500}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'5906","reported_seq":5971,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:41.768475+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:41.768475+0000","last_peered":"2024-09-16T04:50:41.768475+0000","last_clean":"2024-09-16T04:50:41.768475+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:41.768475+0000","last_undegraded":"2024-09-16T04:50:41.768475+0000","last_fullsized":"2024-09-16T04:50:41.768475+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5906,"log_dups_size":0,"ondisk_log_size":5906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24190976,"num_objects":5906,"num_object_clones":0,"num_object_copies":11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5906,"num_write_kb":23624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'5747","reported_seq":5957,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:37.285345+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:37.285345+0000","last_peered":"2024-09-16T04:50:37.285345+0000","last_clean":"2024-09-16T04:50:37.285345+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:37.285345+0000","last_undegraded":"2024-09-16T04:50:37.285345+0000","last_fullsized":"2024-09-16T04:50:37.285345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":5747,"log_dups_size":0,"ondisk_log_size":5747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23416832,"num_objects":5717,"num_object_clones":0,"num_object_copies":11434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2779,"num_write_kb":11116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'5813","reported_seq":5941,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:37.290263+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:37.290263+0000","last_peered":"2024-09-16T04:50:37.290263+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:37.290263+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":5813,"log_dups_size":0,"ondisk_log_size":5813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23789568,"num_objects":5808,"num_object_clones":0,"num_object_copies":11616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2806,"num_write_kb":11224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2352},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'5676","reported_seq":5819,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:37.287855+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:37.287855+0000","last_peered":"2024-09-16T04:50:37.287855+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:37.287855+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":5676,"log_dups_size":0,"ondisk_log_size":5676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23224320,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2331,"num_write_kb":9324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2271},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'5658","reported_seq":5868,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:37.241210+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:37.241210+0000","last_peered":"2024-09-16T04:50:37.241210+0000","last_clean":"2024-09-16T04:50:37.241210+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:37.241210+0000","last_undegraded":"2024-09-16T04:50:37.241210+0000","last_fullsized":"2024-09-16T04:50:37.241210+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":5658,"log_dups_size":0,"ondisk_log_size":5658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23142400,"num_objects":5650,"num_object_clones":0,"num_object_copies":11300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2728,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'5849","reported_seq":6072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.740999+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:41.740999+0000","last_peered":"2024-09-16T04:50:41.740999+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:41.740999+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":5849,"log_dups_size":0,"ondisk_log_size":5849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23822336,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2600,"num_write_kb":10400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":2495}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'5826","reported_seq":5951,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:37.309883+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:37.309883+0000","last_peered":"2024-09-16T04:50:37.309883+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:37.309883+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":5826,"log_dups_size":0,"ondisk_log_size":5826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23830528,"num_objects":5818,"num_object_clones":0,"num_object_copies":11636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2481,"num_write_kb":9924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2353},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'5845","reported_seq":6082,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.820086+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:41.820086+0000","last_peered":"2024-09-16T04:50:41.820086+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:41.820086+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":5845,"log_dups_size":0,"ondisk_log_size":5845,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23815378,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5817,"num_write_kb":23259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":2520}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'5879","reported_seq":8978,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:41.796244+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:41.796244+0000","last_peered":"2024-09-16T04:50:41.796244+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:41.796244+0000","last_undegraded":"2024-09-16T04:50:41.796244+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":5879,"log_dups_size":0,"ondisk_log_size":5879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24055808,"num_objects":5873,"num_object_clones":0,"num_object_copies":11746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1871,"num_objects_unfound":0,"num_objects_dirty":5873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2602,"num_write_kb":10408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1473,"num_bytes_recovered":6033408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1871},{"shards":"2","objects":4002}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'5854","reported_seq":5947,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.807871+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:41.807871+0000","last_peered":"2024-09-16T04:50:41.807871+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:41.807871+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5854,"log_dups_size":0,"ondisk_log_size":5854,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23970409,"num_objects":5853,"num_object_clones":0,"num_object_copies":11706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5853,"num_write_kb":23409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":2446}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'5988","reported_seq":6086,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.832069+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:41.832069+0000","last_peered":"2024-09-16T04:50:41.832069+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:41.832069+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":5988,"log_dups_size":0,"ondisk_log_size":5988,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24495931,"num_objects":5983,"num_object_clones":0,"num_object_copies":11966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5985,"num_write_kb":23924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":2469}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'5726","reported_seq":5852,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:37.326684+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:37.326684+0000","last_peered":"2024-09-16T04:50:37.326684+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:37.326684+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":5726,"log_dups_size":0,"ondisk_log_size":5726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23437312,"num_objects":5722,"num_object_clones":0,"num_object_copies":11444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2480,"num_write_kb":9920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2397},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'5865","reported_seq":6000,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:37.318443+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:37.318443+0000","last_peered":"2024-09-16T04:50:37.318443+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:37.318443+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":5865,"log_dups_size":0,"ondisk_log_size":5865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23974505,"num_objects":5854,"num_object_clones":0,"num_object_copies":11708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2634,"num_write_kb":10536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2330},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":380440799,"num_objects":92887,"num_object_clones":0,"num_object_copies":185774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1871,"num_objects_unfound":0,"num_objects_dirty":92887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58121,"num_write_kb":232456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1482,"num_bytes_recovered":6045696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":767422464,"data_stored":767335489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93043,"ondisk_log_size":93043,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505600,"kb_used_data":317040,"kb_used_omap":4,"kb_used_meta":188539,"kb_avail":93866240,"statfs":{"total":96636764160,"available":96119029760,"internally_reserved":0,"allocated":324648960,"data_stored":324390198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":193064617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593742,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247676,"kb_used_data":121580,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94124164,"statfs":{"total":96636764160,"available":96383143936,"internally_reserved":0,"allocated":124497920,"data_stored":124162620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":129101486},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486476,"kb_used_data":310460,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93885364,"statfs":{"total":96636764160,"available":96138612736,"internally_reserved":0,"allocated":317911040,"data_stored":317652278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":317841408,"data_stored":317837929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":125173760,"data_stored":125093743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324407296,"data_stored":324403817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:44.329 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:45.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 108 20 96073 96053 3.47314 2.19531 0.0671277 0.0224832 2024-09-16T04:50:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:45.236 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:45.306 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-16T04:50:44.298038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":386466034,"num_objects":94360,"num_object_clones":0,"num_object_copies":188720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1871,"num_objects_unfound":0,"num_objects_dirty":94360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59594,"num_write_kb":238342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1483,"num_bytes_recovered":6045715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94517,"ondisk_log_size":94517,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1255404,"kb_used_data":756412,"kb_used_omap":12,"kb_used_meta":498931,"kb_avail":281860116,"statfs":{"total":289910292480,"available":288624758784,"internally_reserved":0,"allocated":774565888,"data_stored":773713064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":510905345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,4,6,14,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30298112,"num_objects":7397,"num_object_clones":0,"num_object_copies":14794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":7397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7397,"num_write_kb":29588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":1159168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002955"},"pg_stats":[{"pgid":"2.f","version":"45'5838","reported_seq":5913,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:42.291919+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:42.291919+0000","last_peered":"2024-09-16T04:50:42.291919+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:42.291919+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5838,"log_dups_size":0,"ondisk_log_size":5838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23912448,"num_objects":5838,"num_object_clones":0,"num_object_copies":11676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5838,"num_write_kb":23352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2497},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'5930","reported_seq":6041,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:41.820396+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:41.820396+0000","last_peered":"2024-09-16T04:50:41.820396+0000","last_clean":"2024-09-16T04:50:41.820396+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:41.820396+0000","last_undegraded":"2024-09-16T04:50:41.820396+0000","last_fullsized":"2024-09-16T04:50:41.820396+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":5930,"log_dups_size":0,"ondisk_log_size":5930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264704,"num_objects":5924,"num_object_clones":0,"num_object_copies":11848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2899,"num_write_kb":11596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'5823","reported_seq":5956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.750987+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:41.750987+0000","last_peered":"2024-09-16T04:50:41.750987+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:41.750987+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":5823,"log_dups_size":0,"ondisk_log_size":5823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23834624,"num_objects":5819,"num_object_clones":0,"num_object_copies":11638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2562,"num_write_kb":10248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":2500}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'5906","reported_seq":5971,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:41.768475+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:41.768475+0000","last_peered":"2024-09-16T04:50:41.768475+0000","last_clean":"2024-09-16T04:50:41.768475+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:41.768475+0000","last_undegraded":"2024-09-16T04:50:41.768475+0000","last_fullsized":"2024-09-16T04:50:41.768475+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5906,"log_dups_size":0,"ondisk_log_size":5906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24190976,"num_objects":5906,"num_object_clones":0,"num_object_copies":11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5906,"num_write_kb":23624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'5934","reported_seq":6148,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:42.258933+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:42.258933+0000","last_peered":"2024-09-16T04:50:42.258933+0000","last_clean":"2024-09-16T04:50:42.258933+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:42.258933+0000","last_undegraded":"2024-09-16T04:50:42.258933+0000","last_fullsized":"2024-09-16T04:50:42.258933+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":5934,"log_dups_size":0,"ondisk_log_size":5934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24182784,"num_objects":5904,"num_object_clones":0,"num_object_copies":11808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2966,"num_write_kb":11864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'5991","reported_seq":6117,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:42.340703+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:42.340703+0000","last_peered":"2024-09-16T04:50:42.340703+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:42.340703+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":5991,"log_dups_size":0,"ondisk_log_size":5991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24518656,"num_objects":5986,"num_object_clones":0,"num_object_copies":11972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2984,"num_write_kb":11936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2530},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'5852","reported_seq":5996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:42.345724+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:42.345724+0000","last_peered":"2024-09-16T04:50:42.345724+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:42.345724+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":5852,"log_dups_size":0,"ondisk_log_size":5852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23945216,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2507,"num_write_kb":10028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2447},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'5845","reported_seq":6056,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:42.265199+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:42.265199+0000","last_peered":"2024-09-16T04:50:42.265199+0000","last_clean":"2024-09-16T04:50:42.265199+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:42.265199+0000","last_undegraded":"2024-09-16T04:50:42.265199+0000","last_fullsized":"2024-09-16T04:50:42.265199+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":5845,"log_dups_size":0,"ondisk_log_size":5845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908352,"num_objects":5837,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2915,"num_write_kb":11660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'5849","reported_seq":6072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.740999+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:41.740999+0000","last_peered":"2024-09-16T04:50:41.740999+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:41.740999+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":5849,"log_dups_size":0,"ondisk_log_size":5849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23822336,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2600,"num_write_kb":10400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":2495}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'6029","reported_seq":6153,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:42.266644+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:42.266644+0000","last_peered":"2024-09-16T04:50:42.266644+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:42.266644+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":6029,"log_dups_size":0,"ondisk_log_size":6029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24662016,"num_objects":6021,"num_object_clones":0,"num_object_copies":12042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2684,"num_write_kb":10736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2556},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'5845","reported_seq":6082,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.820086+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:41.820086+0000","last_peered":"2024-09-16T04:50:41.820086+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:41.820086+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":5845,"log_dups_size":0,"ondisk_log_size":5845,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23815378,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5817,"num_write_kb":23259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":2520}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'5879","reported_seq":8978,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:41.796244+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:41.796244+0000","last_peered":"2024-09-16T04:50:41.796244+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:41.796244+0000","last_undegraded":"2024-09-16T04:50:41.796244+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":5879,"log_dups_size":0,"ondisk_log_size":5879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24055808,"num_objects":5873,"num_object_clones":0,"num_object_copies":11746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1871,"num_objects_unfound":0,"num_objects_dirty":5873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2602,"num_write_kb":10408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1473,"num_bytes_recovered":6033408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1871},{"shards":"2","objects":4002}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'5854","reported_seq":5947,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.807871+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:41.807871+0000","last_peered":"2024-09-16T04:50:41.807871+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:41.807871+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5854,"log_dups_size":0,"ondisk_log_size":5854,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23970409,"num_objects":5853,"num_object_clones":0,"num_object_copies":11706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5853,"num_write_kb":23409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":2446}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'5988","reported_seq":6086,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.832069+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:41.832069+0000","last_peered":"2024-09-16T04:50:41.832069+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:41.832069+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":5988,"log_dups_size":0,"ondisk_log_size":5988,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24495931,"num_objects":5983,"num_object_clones":0,"num_object_copies":11966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5985,"num_write_kb":23924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":2469}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'5907","reported_seq":6034,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:42.222616+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:42.222616+0000","last_peered":"2024-09-16T04:50:42.222616+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:42.222616+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":5907,"log_dups_size":0,"ondisk_log_size":5907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24178688,"num_objects":5903,"num_object_clones":0,"num_object_copies":11806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2661,"num_write_kb":10644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2578},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'6044","reported_seq":6179,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:42.197399+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:42.197399+0000","last_peered":"2024-09-16T04:50:42.197399+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:42.197399+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":6044,"log_dups_size":0,"ondisk_log_size":6044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24707689,"num_objects":6033,"num_object_clones":0,"num_object_copies":12066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2813,"num_write_kb":11252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2509},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":386466015,"num_objects":94358,"num_object_clones":0,"num_object_copies":188716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1871,"num_objects_unfound":0,"num_objects_dirty":94358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59592,"num_write_kb":238340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1482,"num_bytes_recovered":6045696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":774955008,"data_stored":774868033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94514,"ondisk_log_size":94514,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505600,"kb_used_data":317040,"kb_used_omap":4,"kb_used_meta":188539,"kb_avail":93866240,"statfs":{"total":96636764160,"available":96119029760,"internally_reserved":0,"allocated":324648960,"data_stored":324390198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":193064617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593742,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247676,"kb_used_data":121580,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94124164,"statfs":{"total":96636764160,"available":96383143936,"internally_reserved":0,"allocated":124497920,"data_stored":124162620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":129101486},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502128,"kb_used_data":317792,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93869712,"statfs":{"total":96636764160,"available":96122585088,"internally_reserved":0,"allocated":325419008,"data_stored":325160246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,6,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":325373952,"data_stored":325370473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":125173760,"data_stored":125093743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324407296,"data_stored":324403817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:45.307 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:50:45.308 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'5838", 'reported_seq': 5913, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:42.291919+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:50:42.291919+0000', 'last_peered': '2024-09-16T04:50:42.291919+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:50:42.291919+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 5838, 'log_dups_size': 0, 'ondisk_log_size': 5838, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23912448, 'num_objects': 5838, 'num_object_clones': 0, 'num_object_copies': 11676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5838, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5838, 'num_write_kb': 23352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2497}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:45.308 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:50:45.308 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'5823", 'reported_seq': 5956, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:41.750987+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:50:41.750987+0000', 'last_peered': '2024-09-16T04:50:41.750987+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:50:41.750987+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 5823, 'log_dups_size': 0, 'ondisk_log_size': 5823, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23834624, 'num_objects': 5819, 'num_object_clones': 0, 'num_object_copies': 11638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5819, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2562, 'num_write_kb': 10248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 2500}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:45.308 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:50:45.308 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'5991", 'reported_seq': 6117, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:42.340703+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:50:42.340703+0000', 'last_peered': '2024-09-16T04:50:42.340703+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:50:42.340703+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 5991, 'log_dups_size': 0, 'ondisk_log_size': 5991, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24518656, 'num_objects': 5986, 'num_object_clones': 0, 'num_object_copies': 11972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5986, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2984, 'num_write_kb': 11936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2530}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:45.308 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:50:45.308 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'5852", 'reported_seq': 5996, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:42.345724+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:50:42.345724+0000', 'last_peered': '2024-09-16T04:50:42.345724+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:50:42.345724+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 5852, 'log_dups_size': 0, 'ondisk_log_size': 5852, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23945216, 'num_objects': 5846, 'num_object_clones': 0, 'num_object_copies': 11692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5846, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2507, 'num_write_kb': 10028, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2447}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:45.308 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:50:45.308 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'5849", 'reported_seq': 6072, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:41.740999+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:50:41.740999+0000', 'last_peered': '2024-09-16T04:50:41.740999+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:50:41.740999+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 5849, 'log_dups_size': 0, 'ondisk_log_size': 5849, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23822336, 'num_objects': 5816, 'num_object_clones': 0, 'num_object_copies': 11632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5816, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2600, 'num_write_kb': 10400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 2495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:45.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:50:45.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'6029", 'reported_seq': 6153, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:42.266644+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:50:42.266644+0000', 'last_peered': '2024-09-16T04:50:42.266644+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:50:42.266644+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 6029, 'log_dups_size': 0, 'ondisk_log_size': 6029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24662016, 'num_objects': 6021, 'num_object_clones': 0, 'num_object_copies': 12042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6021, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2684, 'num_write_kb': 10736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2556}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:45.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:50:45.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'5845", 'reported_seq': 6082, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:41.820086+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:50:41.820086+0000', 'last_peered': '2024-09-16T04:50:41.820086+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:50:41.820086+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 5845, 'log_dups_size': 0, 'ondisk_log_size': 5845, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23815378, 'num_objects': 5816, 'num_object_clones': 0, 'num_object_copies': 11632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5816, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5817, 'num_write_kb': 23259, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 2520}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:45.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:50:45.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'5879", 'reported_seq': 8978, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:50:41.796244+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:50:41.796244+0000', 'last_peered': '2024-09-16T04:50:41.796244+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:50:41.796244+0000', 'last_undegraded': '2024-09-16T04:50:41.796244+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 5879, 'log_dups_size': 0, 'ondisk_log_size': 5879, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24055808, 'num_objects': 5873, 'num_object_clones': 0, 'num_object_copies': 11746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1871, 'num_objects_unfound': 0, 'num_objects_dirty': 5873, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2602, 'num_write_kb': 10408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1473, 'num_bytes_recovered': 6033408, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1871}, {'shards': '2', 'objects': 4002}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:45.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:50:45.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'5854", 'reported_seq': 5947, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:41.807871+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:50:41.807871+0000', 'last_peered': '2024-09-16T04:50:41.807871+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:50:41.807871+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 5854, 'log_dups_size': 0, 'ondisk_log_size': 5854, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23970409, 'num_objects': 5853, 'num_object_clones': 0, 'num_object_copies': 11706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5853, 'num_write_kb': 23409, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 2446}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:45.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:50:45.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'5988", 'reported_seq': 6086, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:41.832069+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:50:41.832069+0000', 'last_peered': '2024-09-16T04:50:41.832069+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:50:41.832069+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 5988, 'log_dups_size': 0, 'ondisk_log_size': 5988, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24495931, 'num_objects': 5983, 'num_object_clones': 0, 'num_object_copies': 11966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5983, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5985, 'num_write_kb': 23924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 2469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:45.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:50:45.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'5907", 'reported_seq': 6034, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:42.222616+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:50:42.222616+0000', 'last_peered': '2024-09-16T04:50:42.222616+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:50:42.222616+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 5907, 'log_dups_size': 0, 'ondisk_log_size': 5907, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24178688, 'num_objects': 5903, 'num_object_clones': 0, 'num_object_copies': 11806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2661, 'num_write_kb': 10644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2578}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:45.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:50:45.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'6044", 'reported_seq': 6179, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:42.197399+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:50:42.197399+0000', 'last_peered': '2024-09-16T04:50:42.197399+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:50:42.197399+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 6044, 'log_dups_size': 0, 'ondisk_log_size': 6044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24707689, 'num_objects': 6033, 'num_object_clones': 0, 'num_object_copies': 12066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6033, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2813, 'num_write_kb': 11252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2509}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:45.311 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:50:46.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 109 20 96620 96600 3.46088 2.13672 0.0151896 0.022565 2024-09-16T04:50:46.210 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:46.210 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/188720 objects degraded (19.811%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":193,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":94360,"data_bytes":386466034,"bytes_used":1285533696,"bytes_avail":288624758784,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":188720,"degraded_ratio":0.19810830860534126,"misplaced_objects":1871,"misplaced_total":188720,"misplaced_ratio":0.0099141585417549805,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":96573,"recovering_keys_per_sec":0,"num_objects_recovered":283,"num_bytes_recovered":1159168,"num_keys_recovered":0,"write_bytes_sec":2524220,"read_op_per_sec":0,"write_op_per_sec":616},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:50:46.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:50:46.278 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:47.080 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 110 20 97157 97137 3.44848 2.09766 0.0108292 0.0226439 2024-09-16T04:50:47.161 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:47.165 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:47.249 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-16T04:50:46.299907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":386466034,"num_objects":94360,"num_object_clones":0,"num_object_copies":188720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1871,"num_objects_unfound":0,"num_objects_dirty":94360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59594,"num_write_kb":238342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1483,"num_bytes_recovered":6045715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94517,"ondisk_log_size":94517,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1272868,"kb_used_data":765556,"kb_used_omap":12,"kb_used_meta":507251,"kb_avail":281842652,"statfs":{"total":289910292480,"available":288606875648,"internally_reserved":0,"allocated":783929344,"data_stored":783076520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":519425025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,2,4,6,14,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24084480,"num_objects":5880,"num_object_clones":0,"num_object_copies":11760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":5880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5880,"num_write_kb":23520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":1159168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004286"},"pg_stats":[{"pgid":"2.f","version":"45'5838","reported_seq":5913,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:42.291919+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:42.291919+0000","last_peered":"2024-09-16T04:50:42.291919+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:42.291919+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5838,"log_dups_size":0,"ondisk_log_size":5838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23912448,"num_objects":5838,"num_object_clones":0,"num_object_copies":11676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5838,"num_write_kb":23352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2497},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'5930","reported_seq":6041,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:41.820396+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:41.820396+0000","last_peered":"2024-09-16T04:50:41.820396+0000","last_clean":"2024-09-16T04:50:41.820396+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:41.820396+0000","last_undegraded":"2024-09-16T04:50:41.820396+0000","last_fullsized":"2024-09-16T04:50:41.820396+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":5930,"log_dups_size":0,"ondisk_log_size":5930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264704,"num_objects":5924,"num_object_clones":0,"num_object_copies":11848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2899,"num_write_kb":11596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'5823","reported_seq":5956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.750987+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:41.750987+0000","last_peered":"2024-09-16T04:50:41.750987+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:41.750987+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":5823,"log_dups_size":0,"ondisk_log_size":5823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23834624,"num_objects":5819,"num_object_clones":0,"num_object_copies":11638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2562,"num_write_kb":10248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":2500}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'5906","reported_seq":5971,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:41.768475+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:41.768475+0000","last_peered":"2024-09-16T04:50:41.768475+0000","last_clean":"2024-09-16T04:50:41.768475+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:41.768475+0000","last_undegraded":"2024-09-16T04:50:41.768475+0000","last_fullsized":"2024-09-16T04:50:41.768475+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5906,"log_dups_size":0,"ondisk_log_size":5906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24190976,"num_objects":5906,"num_object_clones":0,"num_object_copies":11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5906,"num_write_kb":23624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'5934","reported_seq":6148,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:42.258933+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:42.258933+0000","last_peered":"2024-09-16T04:50:42.258933+0000","last_clean":"2024-09-16T04:50:42.258933+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:42.258933+0000","last_undegraded":"2024-09-16T04:50:42.258933+0000","last_fullsized":"2024-09-16T04:50:42.258933+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":5934,"log_dups_size":0,"ondisk_log_size":5934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24182784,"num_objects":5904,"num_object_clones":0,"num_object_copies":11808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2966,"num_write_kb":11864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'5991","reported_seq":6117,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:42.340703+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:42.340703+0000","last_peered":"2024-09-16T04:50:42.340703+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:42.340703+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":5991,"log_dups_size":0,"ondisk_log_size":5991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24518656,"num_objects":5986,"num_object_clones":0,"num_object_copies":11972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2984,"num_write_kb":11936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2530},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'5852","reported_seq":5996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:42.345724+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:42.345724+0000","last_peered":"2024-09-16T04:50:42.345724+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:42.345724+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":5852,"log_dups_size":0,"ondisk_log_size":5852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23945216,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2507,"num_write_kb":10028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2447},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'5845","reported_seq":6056,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:42.265199+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:42.265199+0000","last_peered":"2024-09-16T04:50:42.265199+0000","last_clean":"2024-09-16T04:50:42.265199+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:42.265199+0000","last_undegraded":"2024-09-16T04:50:42.265199+0000","last_fullsized":"2024-09-16T04:50:42.265199+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":5845,"log_dups_size":0,"ondisk_log_size":5845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908352,"num_objects":5837,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2915,"num_write_kb":11660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'5849","reported_seq":6072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.740999+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:41.740999+0000","last_peered":"2024-09-16T04:50:41.740999+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:41.740999+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":5849,"log_dups_size":0,"ondisk_log_size":5849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23822336,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2600,"num_write_kb":10400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":2495}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'6029","reported_seq":6153,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:42.266644+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:42.266644+0000","last_peered":"2024-09-16T04:50:42.266644+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:42.266644+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":6029,"log_dups_size":0,"ondisk_log_size":6029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24662016,"num_objects":6021,"num_object_clones":0,"num_object_copies":12042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2684,"num_write_kb":10736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2556},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'5845","reported_seq":6082,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.820086+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:41.820086+0000","last_peered":"2024-09-16T04:50:41.820086+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:41.820086+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":5845,"log_dups_size":0,"ondisk_log_size":5845,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23815378,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5817,"num_write_kb":23259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":2520}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'5879","reported_seq":8978,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:41.796244+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:41.796244+0000","last_peered":"2024-09-16T04:50:41.796244+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:41.796244+0000","last_undegraded":"2024-09-16T04:50:41.796244+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":5879,"log_dups_size":0,"ondisk_log_size":5879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24055808,"num_objects":5873,"num_object_clones":0,"num_object_copies":11746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1871,"num_objects_unfound":0,"num_objects_dirty":5873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2602,"num_write_kb":10408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1473,"num_bytes_recovered":6033408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1871},{"shards":"2","objects":4002}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'5854","reported_seq":5947,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.807871+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:41.807871+0000","last_peered":"2024-09-16T04:50:41.807871+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:41.807871+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":5854,"log_dups_size":0,"ondisk_log_size":5854,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23970409,"num_objects":5853,"num_object_clones":0,"num_object_copies":11706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5853,"num_write_kb":23409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":2446}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'5988","reported_seq":6086,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:41.832069+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:41.832069+0000","last_peered":"2024-09-16T04:50:41.832069+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:41.832069+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":5988,"log_dups_size":0,"ondisk_log_size":5988,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24495931,"num_objects":5983,"num_object_clones":0,"num_object_copies":11966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5985,"num_write_kb":23924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":2469}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'5907","reported_seq":6034,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:42.222616+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:42.222616+0000","last_peered":"2024-09-16T04:50:42.222616+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:42.222616+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":5907,"log_dups_size":0,"ondisk_log_size":5907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24178688,"num_objects":5903,"num_object_clones":0,"num_object_copies":11806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2661,"num_write_kb":10644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2578},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'6044","reported_seq":6179,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:42.197399+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:42.197399+0000","last_peered":"2024-09-16T04:50:42.197399+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:42.197399+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":6044,"log_dups_size":0,"ondisk_log_size":6044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24707689,"num_objects":6033,"num_object_clones":0,"num_object_copies":12066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2813,"num_write_kb":11252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2509},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":386466015,"num_objects":94358,"num_object_clones":0,"num_object_copies":188716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1871,"num_objects_unfound":0,"num_objects_dirty":94358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59592,"num_write_kb":238340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1482,"num_bytes_recovered":6045696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":784433152,"data_stored":784346177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94514,"ondisk_log_size":94514,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505600,"kb_used_data":317040,"kb_used_omap":4,"kb_used_meta":188539,"kb_avail":93866240,"statfs":{"total":96636764160,"available":96119029760,"internally_reserved":0,"allocated":324648960,"data_stored":324390198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":193064617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593743,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265140,"kb_used_data":130724,"kb_used_omap":4,"kb_used_meta":134395,"kb_avail":94106700,"statfs":{"total":96636764160,"available":96365260800,"internally_reserved":0,"allocated":133861376,"data_stored":133526076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":137621166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502128,"kb_used_data":317792,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93869712,"statfs":{"total":96636764160,"available":96122585088,"internally_reserved":0,"allocated":325419008,"data_stored":325160246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,6,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":325373952,"data_stored":325370473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":134651904,"data_stored":134571887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324407296,"data_stored":324403817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:48.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 111 20 97740 97720 3.43793 2.27734 0.0433776 0.0227181 2024-09-16T04:50:49.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 112 20 98319 98299 3.42743 2.26172 0.00985898 0.0227847 2024-09-16T04:50:50.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 113 20 98888 98868 3.41677 2.22266 0.0419077 0.0228562 2024-09-16T04:50:50.252 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:51.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 114 20 99400 99380 3.40434 2 0.0479869 0.0229411 2024-09-16T04:50:51.083 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:51.187 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-16T04:50:50.302168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398074098,"num_objects":97194,"num_object_clones":0,"num_object_copies":194388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1731,"num_objects_unfound":0,"num_objects_dirty":97194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62428,"num_write_kb":249678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1623,"num_bytes_recovered":6619155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97351,"ondisk_log_size":97351,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1258744,"kb_used_data":788872,"kb_used_omap":12,"kb_used_meta":469811,"kb_avail":281856776,"statfs":{"total":289910292480,"available":288621338624,"internally_reserved":0,"allocated":807804928,"data_stored":806952104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":481086465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,4,6,7,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23449600,"num_objects":5725,"num_object_clones":0,"num_object_copies":11450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":5725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5725,"num_write_kb":22900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005635"},"pg_stats":[{"pgid":"2.f","version":"45'6017","reported_seq":6092,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.346528+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:47.346528+0000","last_peered":"2024-09-16T04:50:47.346528+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:47.346528+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24645632,"num_objects":6017,"num_object_clones":0,"num_object_copies":12034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6017,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2676},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'6102","reported_seq":6213,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:46.780855+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:46.780855+0000","last_peered":"2024-09-16T04:50:46.780855+0000","last_clean":"2024-09-16T04:50:46.780855+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:46.780855+0000","last_undegraded":"2024-09-16T04:50:46.780855+0000","last_fullsized":"2024-09-16T04:50:46.780855+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":6102,"log_dups_size":0,"ondisk_log_size":6102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24969216,"num_objects":6096,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3071,"num_write_kb":12284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'6012","reported_seq":6145,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:46.841777+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:46.841777+0000","last_peered":"2024-09-16T04:50:46.841777+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:46.841777+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":6012,"log_dups_size":0,"ondisk_log_size":6012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24608768,"num_objects":6008,"num_object_clones":0,"num_object_copies":12016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2751,"num_write_kb":11004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":2689}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'6092","reported_seq":6157,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:46.770064+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:46.770064+0000","last_peered":"2024-09-16T04:50:46.770064+0000","last_clean":"2024-09-16T04:50:46.770064+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:46.770064+0000","last_undegraded":"2024-09-16T04:50:46.770064+0000","last_fullsized":"2024-09-16T04:50:46.770064+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6092,"log_dups_size":0,"ondisk_log_size":6092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24952832,"num_objects":6092,"num_object_clones":0,"num_object_copies":12184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6092,"num_write_kb":24368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'6133","reported_seq":6347,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:47.283498+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:47.283498+0000","last_peered":"2024-09-16T04:50:47.283498+0000","last_clean":"2024-09-16T04:50:47.283498+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:47.283498+0000","last_undegraded":"2024-09-16T04:50:47.283498+0000","last_fullsized":"2024-09-16T04:50:47.283498+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":6133,"log_dups_size":0,"ondisk_log_size":6133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24997888,"num_objects":6103,"num_object_clones":0,"num_object_copies":12206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3165,"num_write_kb":12660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'6146","reported_seq":6272,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.285935+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:47.285935+0000","last_peered":"2024-09-16T04:50:47.285935+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:47.285935+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":6146,"log_dups_size":0,"ondisk_log_size":6146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25153536,"num_objects":6141,"num_object_clones":0,"num_object_copies":12282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3139,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2685},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'6010","reported_seq":6154,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.349189+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:47.349189+0000","last_peered":"2024-09-16T04:50:47.349189+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:47.349189+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":6010,"log_dups_size":0,"ondisk_log_size":6010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24592384,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2665,"num_write_kb":10660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2605},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'6030","reported_seq":6241,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:47.303166+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:47.303166+0000","last_peered":"2024-09-16T04:50:47.303166+0000","last_clean":"2024-09-16T04:50:47.303166+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:47.303166+0000","last_undegraded":"2024-09-16T04:50:47.303166+0000","last_fullsized":"2024-09-16T04:50:47.303166+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24666112,"num_objects":6022,"num_object_clones":0,"num_object_copies":12044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3100,"num_write_kb":12400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'6017","reported_seq":6240,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:46.850705+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:46.850705+0000","last_peered":"2024-09-16T04:50:46.850705+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:46.850705+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24510464,"num_objects":5984,"num_object_clones":0,"num_object_copies":11968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2768,"num_write_kb":11072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":2663}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'6193","reported_seq":6316,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.298769+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:47.298769+0000","last_peered":"2024-09-16T04:50:47.298769+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:47.298769+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":6193,"log_dups_size":0,"ondisk_log_size":6193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2848,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2720},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'6013","reported_seq":6250,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:46.869808+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:46.869808+0000","last_peered":"2024-09-16T04:50:46.869808+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:46.869808+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":6013,"log_dups_size":0,"ondisk_log_size":6013,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24503506,"num_objects":5984,"num_object_clones":0,"num_object_copies":11968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5985,"num_write_kb":23931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":2688}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'6071","reported_seq":9451,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:46.884451+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:46.884451+0000","last_peered":"2024-09-16T04:50:46.884451+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:46.884451+0000","last_undegraded":"2024-09-16T04:50:46.884451+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":6071,"log_dups_size":0,"ondisk_log_size":6071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24842240,"num_objects":6065,"num_object_clones":0,"num_object_copies":12130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1731,"num_objects_unfound":0,"num_objects_dirty":6065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2794,"num_write_kb":11176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1613,"num_bytes_recovered":6606848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1731},{"shards":"2","objects":4334}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6019","reported_seq":6112,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:46.869996+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:46.869996+0000","last_peered":"2024-09-16T04:50:46.869996+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:46.869996+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6019,"log_dups_size":0,"ondisk_log_size":6019,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24646249,"num_objects":6018,"num_object_clones":0,"num_object_copies":12036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6018,"num_write_kb":24069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":2611}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'6186","reported_seq":6284,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:46.853265+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:46.853265+0000","last_peered":"2024-09-16T04:50:46.853265+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:46.853265+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":6186,"log_dups_size":0,"ondisk_log_size":6186,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25306939,"num_objects":6181,"num_object_clones":0,"num_object_copies":12362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6183,"num_write_kb":24716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":2667}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'6094","reported_seq":6220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.384975+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:47.384975+0000","last_peered":"2024-09-16T04:50:47.384975+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:47.384975+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":6094,"log_dups_size":0,"ondisk_log_size":6094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24944640,"num_objects":6090,"num_object_clones":0,"num_object_copies":12180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2848,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2765},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'6213","reported_seq":6348,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.369472+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:47.369472+0000","last_peered":"2024-09-16T04:50:47.369472+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:47.369472+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":6213,"log_dups_size":0,"ondisk_log_size":6213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25399913,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2982,"num_write_kb":11928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2678},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398074079,"num_objects":97192,"num_object_clones":0,"num_object_copies":194384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1731,"num_objects_unfound":0,"num_objects_dirty":97192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62426,"num_write_kb":249676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1622,"num_bytes_recovered":6619136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":808501248,"data_stored":808414273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97348,"ondisk_log_size":97348,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500384,"kb_used_data":324304,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93871456,"statfs":{"total":96636764160,"available":96124370944,"internally_reserved":0,"allocated":332087296,"data_stored":331828534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":180285097},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593744,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":261768,"kb_used_data":139832,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94110072,"statfs":{"total":96636764160,"available":96368713728,"internally_reserved":0,"allocated":143187968,"data_stored":142852668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124841646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496592,"kb_used_data":324736,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93875248,"statfs":{"total":96636764160,"available":96128253952,"internally_reserved":0,"allocated":332529664,"data_stored":332270902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":332554240,"data_stored":332550761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":144048128,"data_stored":143968111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331898880,"data_stored":331895401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:51.189 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:51.951 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:51.952 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:52.008 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-16T04:50:50.302168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398074098,"num_objects":97194,"num_object_clones":0,"num_object_copies":194388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1731,"num_objects_unfound":0,"num_objects_dirty":97194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62428,"num_write_kb":249678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1623,"num_bytes_recovered":6619155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97351,"ondisk_log_size":97351,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1258744,"kb_used_data":788872,"kb_used_omap":12,"kb_used_meta":469811,"kb_avail":281856776,"statfs":{"total":289910292480,"available":288621338624,"internally_reserved":0,"allocated":807804928,"data_stored":806952104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":481086465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,4,6,7,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23449600,"num_objects":5725,"num_object_clones":0,"num_object_copies":11450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":5725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5725,"num_write_kb":22900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005635"},"pg_stats":[{"pgid":"2.f","version":"45'6017","reported_seq":6092,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.346528+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:47.346528+0000","last_peered":"2024-09-16T04:50:47.346528+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:47.346528+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24645632,"num_objects":6017,"num_object_clones":0,"num_object_copies":12034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6017,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2676},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'6102","reported_seq":6213,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:46.780855+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:46.780855+0000","last_peered":"2024-09-16T04:50:46.780855+0000","last_clean":"2024-09-16T04:50:46.780855+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:46.780855+0000","last_undegraded":"2024-09-16T04:50:46.780855+0000","last_fullsized":"2024-09-16T04:50:46.780855+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":6102,"log_dups_size":0,"ondisk_log_size":6102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24969216,"num_objects":6096,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3071,"num_write_kb":12284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'6012","reported_seq":6145,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:46.841777+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:46.841777+0000","last_peered":"2024-09-16T04:50:46.841777+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:46.841777+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":6012,"log_dups_size":0,"ondisk_log_size":6012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24608768,"num_objects":6008,"num_object_clones":0,"num_object_copies":12016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2751,"num_write_kb":11004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":2689}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'6092","reported_seq":6157,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:46.770064+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:46.770064+0000","last_peered":"2024-09-16T04:50:46.770064+0000","last_clean":"2024-09-16T04:50:46.770064+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:46.770064+0000","last_undegraded":"2024-09-16T04:50:46.770064+0000","last_fullsized":"2024-09-16T04:50:46.770064+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6092,"log_dups_size":0,"ondisk_log_size":6092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24952832,"num_objects":6092,"num_object_clones":0,"num_object_copies":12184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6092,"num_write_kb":24368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'6133","reported_seq":6347,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:47.283498+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:47.283498+0000","last_peered":"2024-09-16T04:50:47.283498+0000","last_clean":"2024-09-16T04:50:47.283498+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:47.283498+0000","last_undegraded":"2024-09-16T04:50:47.283498+0000","last_fullsized":"2024-09-16T04:50:47.283498+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":6133,"log_dups_size":0,"ondisk_log_size":6133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24997888,"num_objects":6103,"num_object_clones":0,"num_object_copies":12206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3165,"num_write_kb":12660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'6146","reported_seq":6272,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.285935+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:47.285935+0000","last_peered":"2024-09-16T04:50:47.285935+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:47.285935+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":6146,"log_dups_size":0,"ondisk_log_size":6146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25153536,"num_objects":6141,"num_object_clones":0,"num_object_copies":12282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3139,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2685},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'6010","reported_seq":6154,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.349189+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:47.349189+0000","last_peered":"2024-09-16T04:50:47.349189+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:47.349189+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":6010,"log_dups_size":0,"ondisk_log_size":6010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24592384,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2665,"num_write_kb":10660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2605},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'6030","reported_seq":6241,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:47.303166+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:47.303166+0000","last_peered":"2024-09-16T04:50:47.303166+0000","last_clean":"2024-09-16T04:50:47.303166+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:47.303166+0000","last_undegraded":"2024-09-16T04:50:47.303166+0000","last_fullsized":"2024-09-16T04:50:47.303166+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24666112,"num_objects":6022,"num_object_clones":0,"num_object_copies":12044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3100,"num_write_kb":12400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'6017","reported_seq":6240,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:46.850705+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:46.850705+0000","last_peered":"2024-09-16T04:50:46.850705+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:46.850705+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24510464,"num_objects":5984,"num_object_clones":0,"num_object_copies":11968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2768,"num_write_kb":11072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":2663}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783867+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783867+0000","last_peered":"2024-09-16T04:49:37.783867+0000","last_clean":"2024-09-16T04:49:37.783867+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783867+0000","last_undegraded":"2024-09-16T04:49:37.783867+0000","last_fullsized":"2024-09-16T04:49:37.783867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'6193","reported_seq":6316,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.298769+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:47.298769+0000","last_peered":"2024-09-16T04:50:47.298769+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:47.298769+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":6193,"log_dups_size":0,"ondisk_log_size":6193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2848,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2720},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'6013","reported_seq":6250,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:46.869808+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:46.869808+0000","last_peered":"2024-09-16T04:50:46.869808+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:46.869808+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":6013,"log_dups_size":0,"ondisk_log_size":6013,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24503506,"num_objects":5984,"num_object_clones":0,"num_object_copies":11968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5985,"num_write_kb":23931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":2688}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'6071","reported_seq":9451,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:46.884451+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:46.884451+0000","last_peered":"2024-09-16T04:50:46.884451+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:46.884451+0000","last_undegraded":"2024-09-16T04:50:46.884451+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":6071,"log_dups_size":0,"ondisk_log_size":6071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24842240,"num_objects":6065,"num_object_clones":0,"num_object_copies":12130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1731,"num_objects_unfound":0,"num_objects_dirty":6065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2794,"num_write_kb":11176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1613,"num_bytes_recovered":6606848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1731},{"shards":"2","objects":4334}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6019","reported_seq":6112,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:46.869996+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:46.869996+0000","last_peered":"2024-09-16T04:50:46.869996+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:46.869996+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6019,"log_dups_size":0,"ondisk_log_size":6019,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24646249,"num_objects":6018,"num_object_clones":0,"num_object_copies":12036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6018,"num_write_kb":24069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":2611}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'6186","reported_seq":6284,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:46.853265+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:46.853265+0000","last_peered":"2024-09-16T04:50:46.853265+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:46.853265+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":6186,"log_dups_size":0,"ondisk_log_size":6186,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25306939,"num_objects":6181,"num_object_clones":0,"num_object_copies":12362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6183,"num_write_kb":24716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":2667}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781376+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781376+0000","last_peered":"2024-09-16T04:49:37.781376+0000","last_clean":"2024-09-16T04:49:37.781376+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781376+0000","last_undegraded":"2024-09-16T04:49:37.781376+0000","last_fullsized":"2024-09-16T04:49:37.781376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'6094","reported_seq":6220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.384975+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:47.384975+0000","last_peered":"2024-09-16T04:50:47.384975+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:47.384975+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":6094,"log_dups_size":0,"ondisk_log_size":6094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24944640,"num_objects":6090,"num_object_clones":0,"num_object_copies":12180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2848,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2765},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'6213","reported_seq":6348,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.369472+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:47.369472+0000","last_peered":"2024-09-16T04:50:47.369472+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:47.369472+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":6213,"log_dups_size":0,"ondisk_log_size":6213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25399913,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2982,"num_write_kb":11928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2678},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":398074079,"num_objects":97192,"num_object_clones":0,"num_object_copies":194384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1731,"num_objects_unfound":0,"num_objects_dirty":97192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62426,"num_write_kb":249676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1622,"num_bytes_recovered":6619136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":808501248,"data_stored":808414273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97348,"ondisk_log_size":97348,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500384,"kb_used_data":324304,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93871456,"statfs":{"total":96636764160,"available":96124370944,"internally_reserved":0,"allocated":332087296,"data_stored":331828534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":180285097},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593744,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":261768,"kb_used_data":139832,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94110072,"statfs":{"total":96636764160,"available":96368713728,"internally_reserved":0,"allocated":143187968,"data_stored":142852668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124841646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496592,"kb_used_data":324736,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93875248,"statfs":{"total":96636764160,"available":96128253952,"internally_reserved":0,"allocated":332529664,"data_stored":332270902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":332554240,"data_stored":332550761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":144048128,"data_stored":143968111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331898880,"data_stored":331895401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:52.010 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:50:52.010 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'6017", 'reported_seq': 6092, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:47.346528+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:50:47.346528+0000', 'last_peered': '2024-09-16T04:50:47.346528+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:50:47.346528+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 6017, 'log_dups_size': 0, 'ondisk_log_size': 6017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24645632, 'num_objects': 6017, 'num_object_clones': 0, 'num_object_copies': 12034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6017, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6017, 'num_write_kb': 24068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2676}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:52.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:50:52.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'6012", 'reported_seq': 6145, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:46.841777+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:50:46.841777+0000', 'last_peered': '2024-09-16T04:50:46.841777+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:50:46.841777+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 6012, 'log_dups_size': 0, 'ondisk_log_size': 6012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24608768, 'num_objects': 6008, 'num_object_clones': 0, 'num_object_copies': 12016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6008, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2751, 'num_write_kb': 11004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 2689}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:52.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:50:52.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'6146", 'reported_seq': 6272, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:47.285935+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:50:47.285935+0000', 'last_peered': '2024-09-16T04:50:47.285935+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:50:47.285935+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 6146, 'log_dups_size': 0, 'ondisk_log_size': 6146, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25153536, 'num_objects': 6141, 'num_object_clones': 0, 'num_object_copies': 12282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6141, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3139, 'num_write_kb': 12556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2685}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:52.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:50:52.011 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'6010", 'reported_seq': 6154, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:47.349189+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:50:47.349189+0000', 'last_peered': '2024-09-16T04:50:47.349189+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:50:47.349189+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 6010, 'log_dups_size': 0, 'ondisk_log_size': 6010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24592384, 'num_objects': 6004, 'num_object_clones': 0, 'num_object_copies': 12008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2665, 'num_write_kb': 10660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2605}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:52.012 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:50:52.012 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'6017", 'reported_seq': 6240, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:46.850705+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:50:46.850705+0000', 'last_peered': '2024-09-16T04:50:46.850705+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:50:46.850705+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 6017, 'log_dups_size': 0, 'ondisk_log_size': 6017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24510464, 'num_objects': 5984, 'num_object_clones': 0, 'num_object_copies': 11968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5984, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2768, 'num_write_kb': 11072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 2663}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:52.012 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:50:52.012 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'6193", 'reported_seq': 6316, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:47.298769+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:50:47.298769+0000', 'last_peered': '2024-09-16T04:50:47.298769+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:50:47.298769+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 6193, 'log_dups_size': 0, 'ondisk_log_size': 6193, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25333760, 'num_objects': 6185, 'num_object_clones': 0, 'num_object_copies': 12370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2848, 'num_write_kb': 11392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2720}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:52.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:50:52.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'6013", 'reported_seq': 6250, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:46.869808+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:50:46.869808+0000', 'last_peered': '2024-09-16T04:50:46.869808+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:50:46.869808+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 6013, 'log_dups_size': 0, 'ondisk_log_size': 6013, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24503506, 'num_objects': 5984, 'num_object_clones': 0, 'num_object_copies': 11968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5984, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5985, 'num_write_kb': 23931, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 2688}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:52.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:50:52.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'6071", 'reported_seq': 9451, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:50:46.884451+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:50:46.884451+0000', 'last_peered': '2024-09-16T04:50:46.884451+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:50:46.884451+0000', 'last_undegraded': '2024-09-16T04:50:46.884451+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 6071, 'log_dups_size': 0, 'ondisk_log_size': 6071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24842240, 'num_objects': 6065, 'num_object_clones': 0, 'num_object_copies': 12130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1731, 'num_objects_unfound': 0, 'num_objects_dirty': 6065, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2794, 'num_write_kb': 11176, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1613, 'num_bytes_recovered': 6606848, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1731}, {'shards': '2', 'objects': 4334}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:52.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:50:52.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'6019", 'reported_seq': 6112, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:46.869996+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:50:46.869996+0000', 'last_peered': '2024-09-16T04:50:46.869996+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:50:46.869996+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 6019, 'log_dups_size': 0, 'ondisk_log_size': 6019, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24646249, 'num_objects': 6018, 'num_object_clones': 0, 'num_object_copies': 12036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6018, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6018, 'num_write_kb': 24069, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 2611}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:52.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:50:52.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'6186", 'reported_seq': 6284, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:46.853265+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:50:46.853265+0000', 'last_peered': '2024-09-16T04:50:46.853265+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:50:46.853265+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 6186, 'log_dups_size': 0, 'ondisk_log_size': 6186, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25306939, 'num_objects': 6181, 'num_object_clones': 0, 'num_object_copies': 12362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6181, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6183, 'num_write_kb': 24716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 2667}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:52.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:50:52.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'6094", 'reported_seq': 6220, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:47.384975+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:50:47.384975+0000', 'last_peered': '2024-09-16T04:50:47.384975+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:50:47.384975+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 6094, 'log_dups_size': 0, 'ondisk_log_size': 6094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24944640, 'num_objects': 6090, 'num_object_clones': 0, 'num_object_copies': 12180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6090, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2848, 'num_write_kb': 11392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2765}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:52.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:50:52.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'6213", 'reported_seq': 6348, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:47.369472+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:50:47.369472+0000', 'last_peered': '2024-09-16T04:50:47.369472+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:50:47.369472+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 6213, 'log_dups_size': 0, 'ondisk_log_size': 6213, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25399913, 'num_objects': 6202, 'num_object_clones': 0, 'num_object_copies': 12404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6202, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2982, 'num_write_kb': 11928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2678}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:52.014 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:50:52.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 115 20 99939 99919 3.39305 2.10547 0.0358106 0.0230162 2024-09-16T04:50:52.996 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:52.996 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/197126 objects degraded (18.966%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":200,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":98563,"data_bytes":403681522,"bytes_used":1300393984,"bytes_avail":288609898496,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":197126,"degraded_ratio":0.18966042023883201,"misplaced_objects":1557,"misplaced_total":197126,"misplaced_ratio":0.0078985014660673883,"recovering_objects_per_sec":37,"recovering_bytes_per_sec":152846,"recovering_keys_per_sec":0,"num_objects_recovered":448,"num_bytes_recovered":1835008,"num_keys_recovered":0,"write_bytes_sec":2420291,"read_op_per_sec":0,"write_op_per_sec":590},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:50:53.065 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:50:53.065 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:53.077 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 116 20 100479 100459 3.38198 2.10938 0.0162841 0.0230906 2024-09-16T04:50:53.815 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:53.818 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:53.855 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-16T04:50:52.302643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":403681522,"num_objects":98563,"num_object_clones":0,"num_object_copies":197126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1557,"num_objects_unfound":0,"num_objects_dirty":98563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63797,"num_write_kb":255154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1797,"num_bytes_recovered":7331859,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98720,"ondisk_log_size":98720,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1269916,"kb_used_data":795884,"kb_used_omap":12,"kb_used_meta":473971,"kb_avail":281845604,"statfs":{"total":289910292480,"available":288609898496,"internally_reserved":0,"allocated":814985216,"data_stored":814132392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":485346305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,7,3,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29057024,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-448,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7094,"num_write_kb":28376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":448,"num_bytes_recovered":1835008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005586"},"pg_stats":[{"pgid":"2.f","version":"45'6017","reported_seq":6092,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.346528+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:47.346528+0000","last_peered":"2024-09-16T04:50:47.346528+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:47.346528+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6017,"log_dups_size":0,"ondisk_log_size":6017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24645632,"num_objects":6017,"num_object_clones":0,"num_object_copies":12034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6017,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2676},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'6271","reported_seq":6381,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:51.797244+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:51.797244+0000","last_peered":"2024-09-16T04:50:51.797244+0000","last_clean":"2024-09-16T04:50:51.797244+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:51.797244+0000","last_undegraded":"2024-09-16T04:50:51.797244+0000","last_fullsized":"2024-09-16T04:50:51.797244+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":6271,"log_dups_size":0,"ondisk_log_size":6271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25661440,"num_objects":6265,"num_object_clones":0,"num_object_copies":12530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3240,"num_write_kb":12960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'6202","reported_seq":6335,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:51.792385+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:51.792385+0000","last_peered":"2024-09-16T04:50:51.792385+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:51.792385+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":6202,"log_dups_size":0,"ondisk_log_size":6202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25387008,"num_objects":6198,"num_object_clones":0,"num_object_copies":12396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2941,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":2879}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'6261","reported_seq":6326,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:51.848647+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:51.848647+0000","last_peered":"2024-09-16T04:50:51.848647+0000","last_clean":"2024-09-16T04:50:51.848647+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:51.848647+0000","last_undegraded":"2024-09-16T04:50:51.848647+0000","last_fullsized":"2024-09-16T04:50:51.848647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25645056,"num_objects":6261,"num_object_clones":0,"num_object_copies":12522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6261,"num_write_kb":25044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'6133","reported_seq":6347,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:47.283498+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:47.283498+0000","last_peered":"2024-09-16T04:50:47.283498+0000","last_clean":"2024-09-16T04:50:47.283498+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:47.283498+0000","last_undegraded":"2024-09-16T04:50:47.283498+0000","last_fullsized":"2024-09-16T04:50:47.283498+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":6133,"log_dups_size":0,"ondisk_log_size":6133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24997888,"num_objects":6103,"num_object_clones":0,"num_object_copies":12206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3165,"num_write_kb":12660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'6146","reported_seq":6272,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.285935+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:47.285935+0000","last_peered":"2024-09-16T04:50:47.285935+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:47.285935+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":6146,"log_dups_size":0,"ondisk_log_size":6146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25153536,"num_objects":6141,"num_object_clones":0,"num_object_copies":12282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3139,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2685},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'6010","reported_seq":6154,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.349189+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:47.349189+0000","last_peered":"2024-09-16T04:50:47.349189+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:47.349189+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":6010,"log_dups_size":0,"ondisk_log_size":6010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24592384,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2665,"num_write_kb":10660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2605},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'6030","reported_seq":6241,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:47.303166+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:47.303166+0000","last_peered":"2024-09-16T04:50:47.303166+0000","last_clean":"2024-09-16T04:50:47.303166+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:47.303166+0000","last_undegraded":"2024-09-16T04:50:47.303166+0000","last_fullsized":"2024-09-16T04:50:47.303166+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24666112,"num_objects":6022,"num_object_clones":0,"num_object_copies":12044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3100,"num_write_kb":12400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'6191","reported_seq":6414,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:51.825570+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:51.825570+0000","last_peered":"2024-09-16T04:50:51.825570+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:51.825570+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":6191,"log_dups_size":0,"ondisk_log_size":6191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25223168,"num_objects":6158,"num_object_clones":0,"num_object_copies":12316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2942,"num_write_kb":11768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":2837}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'6193","reported_seq":6316,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.298769+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:47.298769+0000","last_peered":"2024-09-16T04:50:47.298769+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:47.298769+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":6193,"log_dups_size":0,"ondisk_log_size":6193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2848,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2720},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'6183","reported_seq":6420,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:51.824192+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:51.824192+0000","last_peered":"2024-09-16T04:50:51.824192+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:51.824192+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":6183,"log_dups_size":0,"ondisk_log_size":6183,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25199826,"num_objects":6154,"num_object_clones":0,"num_object_copies":12308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6155,"num_write_kb":24611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":2858}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'6233","reported_seq":9961,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:51.877072+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:51.877072+0000","last_peered":"2024-09-16T04:50:51.877072+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:51.877072+0000","last_undegraded":"2024-09-16T04:50:51.877072+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":6233,"log_dups_size":0,"ondisk_log_size":6233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25505792,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1557,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2956,"num_write_kb":11824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1787,"num_bytes_recovered":7319552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1557},{"shards":"2","objects":4670}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6175","reported_seq":6268,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:51.644800+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:51.644800+0000","last_peered":"2024-09-16T04:50:51.644800+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:51.644800+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6175,"log_dups_size":0,"ondisk_log_size":6175,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25285225,"num_objects":6174,"num_object_clones":0,"num_object_copies":12348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6174,"num_write_kb":24693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":2767}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'6365","reported_seq":6463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:51.841738+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:51.841738+0000","last_peered":"2024-09-16T04:50:51.841738+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:51.841738+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":6365,"log_dups_size":0,"ondisk_log_size":6365,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26040123,"num_objects":6360,"num_object_clones":0,"num_object_copies":12720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6362,"num_write_kb":25432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":2846}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'6094","reported_seq":6220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.384975+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:47.384975+0000","last_peered":"2024-09-16T04:50:47.384975+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:47.384975+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":6094,"log_dups_size":0,"ondisk_log_size":6094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24944640,"num_objects":6090,"num_object_clones":0,"num_object_copies":12180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2848,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2765},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'6213","reported_seq":6348,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:47.369472+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:47.369472+0000","last_peered":"2024-09-16T04:50:47.369472+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:47.369472+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":6213,"log_dups_size":0,"ondisk_log_size":6213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25399913,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2982,"num_write_kb":11928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2678},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":403681503,"num_objects":98561,"num_object_clones":0,"num_object_copies":197122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1557,"num_objects_unfound":0,"num_objects_dirty":98561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63795,"num_write_kb":255152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1796,"num_bytes_recovered":7331840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":815546368,"data_stored":815459393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98717,"ondisk_log_size":98717,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511556,"kb_used_data":331316,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93860284,"statfs":{"total":96636764160,"available":96112930816,"internally_reserved":0,"allocated":339267584,"data_stored":339008822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":184544937},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593744,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":261768,"kb_used_data":139832,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94110072,"statfs":{"total":96636764160,"available":96368713728,"internally_reserved":0,"allocated":143187968,"data_stored":142852668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124841646},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":496592,"kb_used_data":324736,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93875248,"statfs":{"total":96636764160,"available":96128253952,"internally_reserved":0,"allocated":332529664,"data_stored":332270902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":332554240,"data_stored":332550761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":144048128,"data_stored":143968111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338944000,"data_stored":338940521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:54.079 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 117 20 101020 101000 3.37114 2.11328 0.0461981 0.0231665 2024-09-16T04:50:55.078 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 118 20 101590 101570 3.36144 2.22656 0.00557045 0.0232311 2024-09-16T04:50:56.078 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 119 20 102170 102150 3.35223 2.26562 0.0127885 0.0232972 2024-09-16T04:50:56.857 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:57.081 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:2024-09-16T04:50:57.080718+0000 min lat: 0.00206892 max lat: 2.69414 avg lat: 0.0233648 2024-09-16T04:50:57.082 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T04:50:57.082 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 120 20 102724 102704 3.34223 2.16406 0.0464736 0.0233648 2024-09-16T04:50:57.552 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:57.554 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-16T04:50:56.308967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":409497842,"num_objects":99983,"num_object_clones":0,"num_object_copies":199966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1557,"num_objects_unfound":0,"num_objects_dirty":99983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65217,"num_write_kb":260834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1797,"num_bytes_recovered":7331859,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100140,"ondisk_log_size":100140,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1302304,"kb_used_data":811632,"kb_used_omap":12,"kb_used_meta":490611,"kb_avail":281813216,"statfs":{"total":289910292480,"available":288576733184,"internally_reserved":0,"allocated":831111168,"data_stored":830258344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":502385665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,0,6,7,12,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23031808,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-314,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5623,"num_write_kb":22492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":1286144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010928"},"pg_stats":[{"pgid":"2.f","version":"45'6175","reported_seq":6250,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:52.457803+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:52.457803+0000","last_peered":"2024-09-16T04:50:52.457803+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:52.457803+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6175,"log_dups_size":0,"ondisk_log_size":6175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25292800,"num_objects":6175,"num_object_clones":0,"num_object_copies":12350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6175,"num_write_kb":24700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2834},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'6271","reported_seq":6381,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:51.797244+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:51.797244+0000","last_peered":"2024-09-16T04:50:51.797244+0000","last_clean":"2024-09-16T04:50:51.797244+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:51.797244+0000","last_undegraded":"2024-09-16T04:50:51.797244+0000","last_fullsized":"2024-09-16T04:50:51.797244+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":6271,"log_dups_size":0,"ondisk_log_size":6271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25661440,"num_objects":6265,"num_object_clones":0,"num_object_copies":12530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3240,"num_write_kb":12960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'6202","reported_seq":6335,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:51.792385+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:51.792385+0000","last_peered":"2024-09-16T04:50:51.792385+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:51.792385+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":6202,"log_dups_size":0,"ondisk_log_size":6202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25387008,"num_objects":6198,"num_object_clones":0,"num_object_copies":12396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2941,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":2879}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'6261","reported_seq":6326,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:51.848647+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:51.848647+0000","last_peered":"2024-09-16T04:50:51.848647+0000","last_clean":"2024-09-16T04:50:51.848647+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:51.848647+0000","last_undegraded":"2024-09-16T04:50:51.848647+0000","last_fullsized":"2024-09-16T04:50:51.848647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6261,"log_dups_size":0,"ondisk_log_size":6261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25645056,"num_objects":6261,"num_object_clones":0,"num_object_copies":12522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6261,"num_write_kb":25044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'6327","reported_seq":6540,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:52.442963+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:52.442963+0000","last_peered":"2024-09-16T04:50:52.442963+0000","last_clean":"2024-09-16T04:50:52.442963+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:52.442963+0000","last_undegraded":"2024-09-16T04:50:52.442963+0000","last_fullsized":"2024-09-16T04:50:52.442963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":6327,"log_dups_size":0,"ondisk_log_size":6327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25792512,"num_objects":6297,"num_object_clones":0,"num_object_copies":12594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3359,"num_write_kb":13436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'6346","reported_seq":6475,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:52.404555+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:52.404555+0000","last_peered":"2024-09-16T04:50:52.404555+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:52.404555+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":6346,"log_dups_size":0,"ondisk_log_size":6346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25972736,"num_objects":6341,"num_object_clones":0,"num_object_copies":12682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3339,"num_write_kb":13356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2885},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'6200","reported_seq":6343,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:52.394127+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:52.394127+0000","last_peered":"2024-09-16T04:50:52.394127+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:52.394127+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":6200,"log_dups_size":0,"ondisk_log_size":6200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25370624,"num_objects":6194,"num_object_clones":0,"num_object_copies":12388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2855,"num_write_kb":11420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2795},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'6191","reported_seq":6402,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:52.371482+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:52.371482+0000","last_peered":"2024-09-16T04:50:52.371482+0000","last_clean":"2024-09-16T04:50:52.371482+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:52.371482+0000","last_undegraded":"2024-09-16T04:50:52.371482+0000","last_fullsized":"2024-09-16T04:50:52.371482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":6191,"log_dups_size":0,"ondisk_log_size":6191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25325568,"num_objects":6183,"num_object_clones":0,"num_object_copies":12366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3261,"num_write_kb":13044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'6191","reported_seq":6414,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:51.825570+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:51.825570+0000","last_peered":"2024-09-16T04:50:51.825570+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:51.825570+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":6191,"log_dups_size":0,"ondisk_log_size":6191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25223168,"num_objects":6158,"num_object_clones":0,"num_object_copies":12316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2942,"num_write_kb":11768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":2837}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'6368","reported_seq":6493,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:52.364658+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:52.364658+0000","last_peered":"2024-09-16T04:50:52.364658+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:52.364658+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":6368,"log_dups_size":0,"ondisk_log_size":6368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26050560,"num_objects":6360,"num_object_clones":0,"num_object_copies":12720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3023,"num_write_kb":12092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2895},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'6183","reported_seq":6420,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:51.824192+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:51.824192+0000","last_peered":"2024-09-16T04:50:51.824192+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:51.824192+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":6183,"log_dups_size":0,"ondisk_log_size":6183,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25199826,"num_objects":6154,"num_object_clones":0,"num_object_copies":12308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6155,"num_write_kb":24611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":2858}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'6233","reported_seq":9961,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:51.877072+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:51.877072+0000","last_peered":"2024-09-16T04:50:51.877072+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:51.877072+0000","last_undegraded":"2024-09-16T04:50:51.877072+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":6233,"log_dups_size":0,"ondisk_log_size":6233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25505792,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1557,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2956,"num_write_kb":11824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1787,"num_bytes_recovered":7319552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1557},{"shards":"2","objects":4670}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6175","reported_seq":6268,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:51.644800+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:51.644800+0000","last_peered":"2024-09-16T04:50:51.644800+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:51.644800+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6175,"log_dups_size":0,"ondisk_log_size":6175,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25285225,"num_objects":6174,"num_object_clones":0,"num_object_copies":12348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6174,"num_write_kb":24693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":2767}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'6365","reported_seq":6463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:51.841738+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:51.841738+0000","last_peered":"2024-09-16T04:50:51.841738+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:51.841738+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":6365,"log_dups_size":0,"ondisk_log_size":6365,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26040123,"num_objects":6360,"num_object_clones":0,"num_object_copies":12720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6362,"num_write_kb":25432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":2846}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'6270","reported_seq":6397,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:52.438089+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:52.438089+0000","last_peered":"2024-09-16T04:50:52.438089+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:52.438089+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":6270,"log_dups_size":0,"ondisk_log_size":6270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25665536,"num_objects":6266,"num_object_clones":0,"num_object_copies":12532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3024,"num_write_kb":12096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2941},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'6379","reported_seq":6514,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:52.448015+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:52.448015+0000","last_peered":"2024-09-16T04:50:52.448015+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:52.448015+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":6379,"log_dups_size":0,"ondisk_log_size":6379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26079849,"num_objects":6368,"num_object_clones":0,"num_object_copies":12736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3148,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2844},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":409497823,"num_objects":99981,"num_object_clones":0,"num_object_copies":199962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1557,"num_objects_unfound":0,"num_objects_dirty":99981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65215,"num_write_kb":260832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1796,"num_bytes_recovered":7331840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":831832064,"data_stored":831745089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100137,"ondisk_log_size":100137,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511556,"kb_used_data":331316,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93860284,"statfs":{"total":96636764160,"available":96112930816,"internally_reserved":0,"allocated":339267584,"data_stored":339008822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":184544937},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593745,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":278904,"kb_used_data":148648,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94092936,"statfs":{"total":96636764160,"available":96351166464,"internally_reserved":0,"allocated":152215552,"data_stored":151880252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":133361326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,3,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511844,"kb_used_data":331668,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93859996,"statfs":{"total":96636764160,"available":96112635904,"internally_reserved":0,"allocated":339628032,"data_stored":339369270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,0,2,4,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":339816448,"data_stored":339812969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":153071616,"data_stored":152991599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338944000,"data_stored":338940521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:57.611 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:50:58.082 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 121 20 103238 103218 3.3312 2.00781 0.0879023 0.0234415 2024-09-16T04:50:58.459 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:58.460 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:50:58.532 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-16T04:50:58.309440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":420958450,"num_objects":102781,"num_object_clones":0,"num_object_copies":205562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1393,"num_objects_unfound":0,"num_objects_dirty":102781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68015,"num_write_kb":272026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1963,"num_bytes_recovered":8003603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102938,"ondisk_log_size":102938,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1312136,"kb_used_data":825624,"kb_used_omap":12,"kb_used_meta":486451,"kb_avail":281803384,"statfs":{"total":289910292480,"available":288566665216,"internally_reserved":0,"allocated":845438976,"data_stored":844586152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":498125825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,9,10,6,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34492416,"num_objects":8421,"num_object_clones":0,"num_object_copies":16842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-478,"num_objects_unfound":0,"num_objects_dirty":8421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8421,"num_write_kb":33684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":1957888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009533"},"pg_stats":[{"pgid":"2.f","version":"45'6343","reported_seq":6418,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:57.383775+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:57.383775+0000","last_peered":"2024-09-16T04:50:57.383775+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:57.383775+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6343,"log_dups_size":0,"ondisk_log_size":6343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25980928,"num_objects":6343,"num_object_clones":0,"num_object_copies":12686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6343,"num_write_kb":25372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3002},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'6457","reported_seq":6568,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:56.872486+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:56.872486+0000","last_peered":"2024-09-16T04:50:56.872486+0000","last_clean":"2024-09-16T04:50:56.872486+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:56.872486+0000","last_undegraded":"2024-09-16T04:50:56.872486+0000","last_fullsized":"2024-09-16T04:50:56.872486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":6457,"log_dups_size":0,"ondisk_log_size":6457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26423296,"num_objects":6451,"num_object_clones":0,"num_object_copies":12902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3426,"num_write_kb":13704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'6375","reported_seq":6508,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:56.798538+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:56.798538+0000","last_peered":"2024-09-16T04:50:56.798538+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:56.798538+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":6375,"log_dups_size":0,"ondisk_log_size":6375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26095616,"num_objects":6371,"num_object_clones":0,"num_object_copies":12742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3114,"num_write_kb":12456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":3052}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'6460","reported_seq":6524,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:56.863772+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:56.863772+0000","last_peered":"2024-09-16T04:50:56.863772+0000","last_clean":"2024-09-16T04:50:56.863772+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:56.863772+0000","last_undegraded":"2024-09-16T04:50:56.863772+0000","last_fullsized":"2024-09-16T04:50:56.863772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6460,"log_dups_size":0,"ondisk_log_size":6460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26460160,"num_objects":6460,"num_object_clones":0,"num_object_copies":12920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6460,"num_write_kb":25840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'6495","reported_seq":6707,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:57.413788+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:57.413788+0000","last_peered":"2024-09-16T04:50:57.413788+0000","last_clean":"2024-09-16T04:50:57.413788+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:57.413788+0000","last_undegraded":"2024-09-16T04:50:57.413788+0000","last_fullsized":"2024-09-16T04:50:57.413788+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":6495,"log_dups_size":0,"ondisk_log_size":6495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26480640,"num_objects":6465,"num_object_clones":0,"num_object_copies":12930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3527,"num_write_kb":14108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'6507","reported_seq":6636,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:57.363773+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:57.363773+0000","last_peered":"2024-09-16T04:50:57.363773+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:57.363773+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":6507,"log_dups_size":0,"ondisk_log_size":6507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26632192,"num_objects":6502,"num_object_clones":0,"num_object_copies":13004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3500,"num_write_kb":14000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3046},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'6395","reported_seq":6538,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:57.451755+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:57.451755+0000","last_peered":"2024-09-16T04:50:57.451755+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:57.451755+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":6395,"log_dups_size":0,"ondisk_log_size":6395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26169344,"num_objects":6389,"num_object_clones":0,"num_object_copies":12778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3050,"num_write_kb":12200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2990},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'6361","reported_seq":6572,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:57.438241+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:57.438241+0000","last_peered":"2024-09-16T04:50:57.438241+0000","last_clean":"2024-09-16T04:50:57.438241+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:57.438241+0000","last_undegraded":"2024-09-16T04:50:57.438241+0000","last_fullsized":"2024-09-16T04:50:57.438241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":6361,"log_dups_size":0,"ondisk_log_size":6361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26021888,"num_objects":6353,"num_object_clones":0,"num_object_copies":12706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3431,"num_write_kb":13724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'6377","reported_seq":6600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:56.828164+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:56.828164+0000","last_peered":"2024-09-16T04:50:56.828164+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:56.828164+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":6377,"log_dups_size":0,"ondisk_log_size":6377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25985024,"num_objects":6344,"num_object_clones":0,"num_object_copies":12688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3128,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":3023}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'6533","reported_seq":6657,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:57.479629+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:57.479629+0000","last_peered":"2024-09-16T04:50:57.479629+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:57.479629+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":6533,"log_dups_size":0,"ondisk_log_size":6533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26726400,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3188,"num_write_kb":12752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3060},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'6355","reported_seq":6592,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:56.845503+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:56.845503+0000","last_peered":"2024-09-16T04:50:56.845503+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:56.845503+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":6355,"log_dups_size":0,"ondisk_log_size":6355,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25904338,"num_objects":6326,"num_object_clones":0,"num_object_copies":12652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6327,"num_write_kb":25299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":3030}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'6408","reported_seq":10468,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:56.880568+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:56.880568+0000","last_peered":"2024-09-16T04:50:56.880568+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:56.880568+0000","last_undegraded":"2024-09-16T04:50:56.880568+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":6408,"log_dups_size":0,"ondisk_log_size":6408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26222592,"num_objects":6402,"num_object_clones":0,"num_object_copies":12804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1393,"num_objects_unfound":0,"num_objects_dirty":6402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3131,"num_write_kb":12524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1953,"num_bytes_recovered":7991296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1393},{"shards":"2","objects":5009}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6314","reported_seq":6405,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:56.895557+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:56.895557+0000","last_peered":"2024-09-16T04:50:56.895557+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:56.895557+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6314,"log_dups_size":0,"ondisk_log_size":6314,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25854569,"num_objects":6313,"num_object_clones":0,"num_object_copies":12626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6313,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":2906}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'6549","reported_seq":6647,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:56.840406+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:56.840406+0000","last_peered":"2024-09-16T04:50:56.840406+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:56.840406+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":6549,"log_dups_size":0,"ondisk_log_size":6549,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26793787,"num_objects":6544,"num_object_clones":0,"num_object_copies":13088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6546,"num_write_kb":26168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":3030}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'6451","reported_seq":6575,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:57.463628+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:57.463628+0000","last_peered":"2024-09-16T04:50:57.463628+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:57.463628+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":6451,"log_dups_size":0,"ondisk_log_size":6451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26406912,"num_objects":6447,"num_object_clones":0,"num_object_copies":12894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3205,"num_write_kb":12820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3122},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'6555","reported_seq":6690,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:57.415280+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:57.415280+0000","last_peered":"2024-09-16T04:50:57.415280+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:57.415280+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":6555,"log_dups_size":0,"ondisk_log_size":6555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26800745,"num_objects":6544,"num_object_clones":0,"num_object_copies":13088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3324,"num_write_kb":13296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3020},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":420958431,"num_objects":102779,"num_object_clones":0,"num_object_copies":205558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1393,"num_objects_unfound":0,"num_objects_dirty":102779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68013,"num_write_kb":272024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1962,"num_bytes_recovered":8003584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":846180352,"data_stored":846093377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102935,"ondisk_log_size":102935,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510264,"kb_used_data":338344,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93861576,"statfs":{"total":96636764160,"available":96114253824,"internally_reserved":0,"allocated":346464256,"data_stored":346205494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":176025257},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,3,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593745,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":278904,"kb_used_data":148648,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94092936,"statfs":{"total":96636764160,"available":96351166464,"internally_reserved":0,"allocated":152215552,"data_stored":151880252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":133361326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,3,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522968,"kb_used_data":338632,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93848872,"statfs":{"total":96636764160,"available":96101244928,"internally_reserved":0,"allocated":346759168,"data_stored":346500406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,6,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347021312,"data_stored":347017833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":153071616,"data_stored":152991599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346087424,"data_stored":346083945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:50:58.533 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:50:58.533 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'6343", 'reported_seq': 6418, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:57.383775+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:50:57.383775+0000', 'last_peered': '2024-09-16T04:50:57.383775+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:50:57.383775+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 6343, 'log_dups_size': 0, 'ondisk_log_size': 6343, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25980928, 'num_objects': 6343, 'num_object_clones': 0, 'num_object_copies': 12686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6343, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6343, 'num_write_kb': 25372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3002}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:58.533 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:50:58.533 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'6375", 'reported_seq': 6508, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:56.798538+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:50:56.798538+0000', 'last_peered': '2024-09-16T04:50:56.798538+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:50:56.798538+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 6375, 'log_dups_size': 0, 'ondisk_log_size': 6375, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26095616, 'num_objects': 6371, 'num_object_clones': 0, 'num_object_copies': 12742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6371, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3114, 'num_write_kb': 12456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 3052}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:58.533 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:50:58.533 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'6507", 'reported_seq': 6636, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:57.363773+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:50:57.363773+0000', 'last_peered': '2024-09-16T04:50:57.363773+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:50:57.363773+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 6507, 'log_dups_size': 0, 'ondisk_log_size': 6507, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26632192, 'num_objects': 6502, 'num_object_clones': 0, 'num_object_copies': 13004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3500, 'num_write_kb': 14000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3046}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:58.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:50:58.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'6395", 'reported_seq': 6538, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:57.451755+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:50:57.451755+0000', 'last_peered': '2024-09-16T04:50:57.451755+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:50:57.451755+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 6395, 'log_dups_size': 0, 'ondisk_log_size': 6395, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26169344, 'num_objects': 6389, 'num_object_clones': 0, 'num_object_copies': 12778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6389, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3050, 'num_write_kb': 12200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2990}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:58.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:50:58.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'6377", 'reported_seq': 6600, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:56.828164+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:50:56.828164+0000', 'last_peered': '2024-09-16T04:50:56.828164+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:50:56.828164+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 6377, 'log_dups_size': 0, 'ondisk_log_size': 6377, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25985024, 'num_objects': 6344, 'num_object_clones': 0, 'num_object_copies': 12688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6344, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3128, 'num_write_kb': 12512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 3023}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:58.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:50:58.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'6533", 'reported_seq': 6657, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:57.479629+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:50:57.479629+0000', 'last_peered': '2024-09-16T04:50:57.479629+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:50:57.479629+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 6533, 'log_dups_size': 0, 'ondisk_log_size': 6533, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26726400, 'num_objects': 6525, 'num_object_clones': 0, 'num_object_copies': 13050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6525, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3188, 'num_write_kb': 12752, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3060}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:58.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:50:58.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'6355", 'reported_seq': 6592, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:56.845503+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:50:56.845503+0000', 'last_peered': '2024-09-16T04:50:56.845503+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:50:56.845503+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 6355, 'log_dups_size': 0, 'ondisk_log_size': 6355, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25904338, 'num_objects': 6326, 'num_object_clones': 0, 'num_object_copies': 12652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6326, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6327, 'num_write_kb': 25299, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 3030}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:58.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:50:58.534 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'6408", 'reported_seq': 10468, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:50:56.880568+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:50:56.880568+0000', 'last_peered': '2024-09-16T04:50:56.880568+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:50:56.880568+0000', 'last_undegraded': '2024-09-16T04:50:56.880568+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 6408, 'log_dups_size': 0, 'ondisk_log_size': 6408, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26222592, 'num_objects': 6402, 'num_object_clones': 0, 'num_object_copies': 12804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1393, 'num_objects_unfound': 0, 'num_objects_dirty': 6402, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3131, 'num_write_kb': 12524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1953, 'num_bytes_recovered': 7991296, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1393}, {'shards': '2', 'objects': 5009}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:58.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:50:58.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'6314", 'reported_seq': 6405, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:56.895557+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:50:56.895557+0000', 'last_peered': '2024-09-16T04:50:56.895557+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:50:56.895557+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 6314, 'log_dups_size': 0, 'ondisk_log_size': 6314, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25854569, 'num_objects': 6313, 'num_object_clones': 0, 'num_object_copies': 12626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6313, 'num_write_kb': 25249, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 2906}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:58.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:50:58.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'6549", 'reported_seq': 6647, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:56.840406+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:50:56.840406+0000', 'last_peered': '2024-09-16T04:50:56.840406+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:50:56.840406+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 6549, 'log_dups_size': 0, 'ondisk_log_size': 6549, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26793787, 'num_objects': 6544, 'num_object_clones': 0, 'num_object_copies': 13088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6546, 'num_write_kb': 26168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 3030}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:50:58.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:50:58.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'6451", 'reported_seq': 6575, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:57.463628+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:50:57.463628+0000', 'last_peered': '2024-09-16T04:50:57.463628+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:50:57.463628+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 6451, 'log_dups_size': 0, 'ondisk_log_size': 6451, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26406912, 'num_objects': 6447, 'num_object_clones': 0, 'num_object_copies': 12894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6447, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3205, 'num_write_kb': 12820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3122}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:58.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:50:58.535 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'6555", 'reported_seq': 6690, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:50:57.415280+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:50:57.415280+0000', 'last_peered': '2024-09-16T04:50:57.415280+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:50:57.415280+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 6555, 'log_dups_size': 0, 'ondisk_log_size': 6555, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26800745, 'num_objects': 6544, 'num_object_clones': 0, 'num_object_copies': 13088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3324, 'num_write_kb': 13296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3020}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:50:58.536 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:50:59.082 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 122 20 103765 103745 3.32077 2.05859 0.0516838 0.0235173 2024-09-16T04:50:59.591 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:50:59.592 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/205562 objects degraded (18.188%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":207,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":102781,"data_bytes":420958450,"bytes_used":1343627264,"bytes_avail":288566665216,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":205562,"degraded_ratio":0.18187700061295373,"misplaced_objects":1393,"misplaced_total":205562,"misplaced_ratio":0.0067765443029353676,"recovering_objects_per_sec":39,"recovering_bytes_per_sec":163027,"recovering_keys_per_sec":0,"num_objects_recovered":480,"num_bytes_recovered":1957888,"num_keys_recovered":0,"write_bytes_sec":2872086,"read_op_per_sec":0,"write_op_per_sec":701},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:50:59.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:50:59.649 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:00.084 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 123 20 104300 104280 3.31071 2.08984 0.0333537 0.0235887 2024-09-16T04:51:00.630 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:00.631 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:00.715 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-16T04:51:00.309956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":420958450,"num_objects":102781,"num_object_clones":0,"num_object_copies":205562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1393,"num_objects_unfound":0,"num_objects_dirty":102781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68015,"num_write_kb":272026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1963,"num_bytes_recovered":8003603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102938,"ondisk_log_size":102938,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1330456,"kb_used_data":834216,"kb_used_omap":12,"kb_used_meta":496179,"kb_avail":281785064,"statfs":{"total":289910292480,"available":288547905536,"internally_reserved":0,"allocated":854237184,"data_stored":853391318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":508087297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,4,7,11,10,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22884352,"num_objects":5587,"num_object_clones":0,"num_object_copies":11174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-338,"num_objects_unfound":0,"num_objects_dirty":5587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5587,"num_write_kb":22348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":1384448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008334"},"pg_stats":[{"pgid":"2.f","version":"45'6343","reported_seq":6418,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:57.383775+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:50:57.383775+0000","last_peered":"2024-09-16T04:50:57.383775+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:50:57.383775+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6343,"log_dups_size":0,"ondisk_log_size":6343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25980928,"num_objects":6343,"num_object_clones":0,"num_object_copies":12686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6343,"num_write_kb":25372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3002},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'6457","reported_seq":6568,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:56.872486+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:50:56.872486+0000","last_peered":"2024-09-16T04:50:56.872486+0000","last_clean":"2024-09-16T04:50:56.872486+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:50:56.872486+0000","last_undegraded":"2024-09-16T04:50:56.872486+0000","last_fullsized":"2024-09-16T04:50:56.872486+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":6457,"log_dups_size":0,"ondisk_log_size":6457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26423296,"num_objects":6451,"num_object_clones":0,"num_object_copies":12902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3426,"num_write_kb":13704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'6375","reported_seq":6508,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:56.798538+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:50:56.798538+0000","last_peered":"2024-09-16T04:50:56.798538+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:50:56.798538+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":6375,"log_dups_size":0,"ondisk_log_size":6375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26095616,"num_objects":6371,"num_object_clones":0,"num_object_copies":12742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3114,"num_write_kb":12456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":3052}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'6460","reported_seq":6524,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:56.863772+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:50:56.863772+0000","last_peered":"2024-09-16T04:50:56.863772+0000","last_clean":"2024-09-16T04:50:56.863772+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:50:56.863772+0000","last_undegraded":"2024-09-16T04:50:56.863772+0000","last_fullsized":"2024-09-16T04:50:56.863772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6460,"log_dups_size":0,"ondisk_log_size":6460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26460160,"num_objects":6460,"num_object_clones":0,"num_object_copies":12920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6460,"num_write_kb":25840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'6495","reported_seq":6707,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:57.413788+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:50:57.413788+0000","last_peered":"2024-09-16T04:50:57.413788+0000","last_clean":"2024-09-16T04:50:57.413788+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:50:57.413788+0000","last_undegraded":"2024-09-16T04:50:57.413788+0000","last_fullsized":"2024-09-16T04:50:57.413788+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":6495,"log_dups_size":0,"ondisk_log_size":6495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26480640,"num_objects":6465,"num_object_clones":0,"num_object_copies":12930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3527,"num_write_kb":14108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'6507","reported_seq":6636,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:57.363773+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:50:57.363773+0000","last_peered":"2024-09-16T04:50:57.363773+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:50:57.363773+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":6507,"log_dups_size":0,"ondisk_log_size":6507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26632192,"num_objects":6502,"num_object_clones":0,"num_object_copies":13004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3500,"num_write_kb":14000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3046},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'6395","reported_seq":6538,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:57.451755+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:50:57.451755+0000","last_peered":"2024-09-16T04:50:57.451755+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:50:57.451755+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":6395,"log_dups_size":0,"ondisk_log_size":6395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26169344,"num_objects":6389,"num_object_clones":0,"num_object_copies":12778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3050,"num_write_kb":12200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2990},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'6361","reported_seq":6572,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:50:57.438241+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:50:57.438241+0000","last_peered":"2024-09-16T04:50:57.438241+0000","last_clean":"2024-09-16T04:50:57.438241+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:50:57.438241+0000","last_undegraded":"2024-09-16T04:50:57.438241+0000","last_fullsized":"2024-09-16T04:50:57.438241+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":6361,"log_dups_size":0,"ondisk_log_size":6361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26021888,"num_objects":6353,"num_object_clones":0,"num_object_copies":12706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3431,"num_write_kb":13724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'6377","reported_seq":6600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:56.828164+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:50:56.828164+0000","last_peered":"2024-09-16T04:50:56.828164+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:50:56.828164+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":6377,"log_dups_size":0,"ondisk_log_size":6377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25985024,"num_objects":6344,"num_object_clones":0,"num_object_copies":12688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3128,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":3023}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'6533","reported_seq":6657,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:57.479629+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:50:57.479629+0000","last_peered":"2024-09-16T04:50:57.479629+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:50:57.479629+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":6533,"log_dups_size":0,"ondisk_log_size":6533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26726400,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3188,"num_write_kb":12752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3060},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'6355","reported_seq":6592,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:56.845503+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:50:56.845503+0000","last_peered":"2024-09-16T04:50:56.845503+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:50:56.845503+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":6355,"log_dups_size":0,"ondisk_log_size":6355,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25904338,"num_objects":6326,"num_object_clones":0,"num_object_copies":12652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6327,"num_write_kb":25299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":3030}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'6408","reported_seq":10468,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:50:56.880568+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:50:56.880568+0000","last_peered":"2024-09-16T04:50:56.880568+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:50:56.880568+0000","last_undegraded":"2024-09-16T04:50:56.880568+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":6408,"log_dups_size":0,"ondisk_log_size":6408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26222592,"num_objects":6402,"num_object_clones":0,"num_object_copies":12804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1393,"num_objects_unfound":0,"num_objects_dirty":6402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3131,"num_write_kb":12524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1953,"num_bytes_recovered":7991296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1393},{"shards":"2","objects":5009}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6314","reported_seq":6405,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:56.895557+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:50:56.895557+0000","last_peered":"2024-09-16T04:50:56.895557+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:50:56.895557+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6314,"log_dups_size":0,"ondisk_log_size":6314,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25854569,"num_objects":6313,"num_object_clones":0,"num_object_copies":12626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6313,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":2906}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'6549","reported_seq":6647,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:56.840406+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:50:56.840406+0000","last_peered":"2024-09-16T04:50:56.840406+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:50:56.840406+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":6549,"log_dups_size":0,"ondisk_log_size":6549,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26793787,"num_objects":6544,"num_object_clones":0,"num_object_copies":13088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6546,"num_write_kb":26168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":3030}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'6451","reported_seq":6575,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:57.463628+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:50:57.463628+0000","last_peered":"2024-09-16T04:50:57.463628+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:50:57.463628+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":6451,"log_dups_size":0,"ondisk_log_size":6451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26406912,"num_objects":6447,"num_object_clones":0,"num_object_copies":12894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3205,"num_write_kb":12820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3122},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'6555","reported_seq":6690,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:50:57.415280+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:50:57.415280+0000","last_peered":"2024-09-16T04:50:57.415280+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:50:57.415280+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":6555,"log_dups_size":0,"ondisk_log_size":6555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26800745,"num_objects":6544,"num_object_clones":0,"num_object_copies":13088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3324,"num_write_kb":13296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3020},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":420958431,"num_objects":102779,"num_object_clones":0,"num_object_copies":205558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1393,"num_objects_unfound":0,"num_objects_dirty":102779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68013,"num_write_kb":272024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1962,"num_bytes_recovered":8003584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":855142400,"data_stored":855062383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102935,"ondisk_log_size":102935,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510264,"kb_used_data":338344,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93861576,"statfs":{"total":96636764160,"available":96114253824,"internally_reserved":0,"allocated":346464256,"data_stored":346205494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":176025257},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,3,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593746,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297224,"kb_used_data":157240,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94074616,"statfs":{"total":96636764160,"available":96332406784,"internally_reserved":0,"allocated":161013760,"data_stored":160685418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522968,"kb_used_data":338632,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93848872,"statfs":{"total":96636764160,"available":96101244928,"internally_reserved":0,"allocated":346759168,"data_stored":346500406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,6,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347021312,"data_stored":347017833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162033664,"data_stored":161960605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":346087424,"data_stored":346083945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:01.084 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 124 20 104845 104825 3.30118 2.12891 0.0591137 0.0236583 2024-09-16T04:51:02.084 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 125 20 105404 105384 3.29224 2.18359 0.0496363 0.023722 2024-09-16T04:51:03.084 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 126 20 105963 105943 3.28344 2.18359 0.0292901 0.0237854 2024-09-16T04:51:03.717 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:04.087 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 127 20 106525 106505 3.27488 2.19531 0.0125183 0.0238482 2024-09-16T04:51:04.522 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:04.596 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-16T04:51:04.312093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":431935730,"num_objects":105461,"num_object_clones":0,"num_object_copies":210922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1235,"num_objects_unfound":0,"num_objects_dirty":105461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70695,"num_write_kb":282746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2121,"num_bytes_recovered":8650771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105618,"ondisk_log_size":105618,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1334188,"kb_used_data":846268,"kb_used_omap":12,"kb_used_meta":487859,"kb_avail":281781332,"statfs":{"total":289910292480,"available":288544083968,"internally_reserved":0,"allocated":866578432,"data_stored":865732566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":499567617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,5,5,12,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28254208,"num_objects":6898,"num_object_clones":0,"num_object_copies":13796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-322,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6898,"num_write_kb":27592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":1318912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009449"},"pg_stats":[{"pgid":"2.f","version":"45'6492","reported_seq":6567,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.428781+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:02.428781+0000","last_peered":"2024-09-16T04:51:02.428781+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:02.428781+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6492,"log_dups_size":0,"ondisk_log_size":6492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26591232,"num_objects":6492,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6492,"num_write_kb":25968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3151},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'6611","reported_seq":6722,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:01.917290+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:01.917290+0000","last_peered":"2024-09-16T04:51:01.917290+0000","last_clean":"2024-09-16T04:51:01.917290+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:01.917290+0000","last_undegraded":"2024-09-16T04:51:01.917290+0000","last_fullsized":"2024-09-16T04:51:01.917290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":6611,"log_dups_size":0,"ondisk_log_size":6611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27054080,"num_objects":6605,"num_object_clones":0,"num_object_copies":13210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3580,"num_write_kb":14320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'6552","reported_seq":6686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.927128+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:01.927128+0000","last_peered":"2024-09-16T04:51:01.927128+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:01.927128+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":6552,"log_dups_size":0,"ondisk_log_size":6552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26820608,"num_objects":6548,"num_object_clones":0,"num_object_copies":13096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3291,"num_write_kb":13164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":3229}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'6616","reported_seq":6679,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:01.858379+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:01.858379+0000","last_peered":"2024-09-16T04:51:01.858379+0000","last_clean":"2024-09-16T04:51:01.858379+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:01.858379+0000","last_undegraded":"2024-09-16T04:51:01.858379+0000","last_fullsized":"2024-09-16T04:51:01.858379+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6616,"log_dups_size":0,"ondisk_log_size":6616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27099136,"num_objects":6616,"num_object_clones":0,"num_object_copies":13232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6616,"num_write_kb":26464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'6676","reported_seq":6890,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:02.299699+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:02.299699+0000","last_peered":"2024-09-16T04:51:02.299699+0000","last_clean":"2024-09-16T04:51:02.299699+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:02.299699+0000","last_undegraded":"2024-09-16T04:51:02.299699+0000","last_fullsized":"2024-09-16T04:51:02.299699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":6676,"log_dups_size":0,"ondisk_log_size":6676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27222016,"num_objects":6646,"num_object_clones":0,"num_object_copies":13292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3708,"num_write_kb":14832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'6684","reported_seq":6813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.411862+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:02.411862+0000","last_peered":"2024-09-16T04:51:02.411862+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:02.411862+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":6684,"log_dups_size":0,"ondisk_log_size":6684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27357184,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3677,"num_write_kb":14708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3223},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'6553","reported_seq":6696,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.465656+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:02.465656+0000","last_peered":"2024-09-16T04:51:02.465656+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:02.465656+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":6553,"log_dups_size":0,"ondisk_log_size":6553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26816512,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3208,"num_write_kb":12832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3148},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'6555","reported_seq":6766,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:02.408635+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:02.408635+0000","last_peered":"2024-09-16T04:51:02.408635+0000","last_clean":"2024-09-16T04:51:02.408635+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:02.408635+0000","last_undegraded":"2024-09-16T04:51:02.408635+0000","last_fullsized":"2024-09-16T04:51:02.408635+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":6555,"log_dups_size":0,"ondisk_log_size":6555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26816512,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3625,"num_write_kb":14500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'6541","reported_seq":6764,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.881759+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:01.881759+0000","last_peered":"2024-09-16T04:51:01.881759+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:01.881759+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":6541,"log_dups_size":0,"ondisk_log_size":6541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26656768,"num_objects":6508,"num_object_clones":0,"num_object_copies":13016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3292,"num_write_kb":13168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":3187}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'6689","reported_seq":6814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.343010+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:02.343010+0000","last_peered":"2024-09-16T04:51:02.343010+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:02.343010+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":6689,"log_dups_size":0,"ondisk_log_size":6689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27365376,"num_objects":6681,"num_object_clones":0,"num_object_copies":13362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3344,"num_write_kb":13376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3216},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'6513","reported_seq":6750,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.849689+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:01.849689+0000","last_peered":"2024-09-16T04:51:01.849689+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:01.849689+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":6513,"log_dups_size":0,"ondisk_log_size":6513,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26551506,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6485,"num_write_kb":25931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":3188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'6568","reported_seq":10943,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:51:01.904889+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:51:01.904889+0000","last_peered":"2024-09-16T04:51:01.904889+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:01.904889+0000","last_undegraded":"2024-09-16T04:51:01.904889+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":6568,"log_dups_size":0,"ondisk_log_size":6568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26877952,"num_objects":6562,"num_object_clones":0,"num_object_copies":13124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1235,"num_objects_unfound":0,"num_objects_dirty":6562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3291,"num_write_kb":13164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2111,"num_bytes_recovered":8638464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1235},{"shards":"2","objects":5327}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6479","reported_seq":6570,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.904885+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:01.904885+0000","last_peered":"2024-09-16T04:51:01.904885+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:01.904885+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6479,"log_dups_size":0,"ondisk_log_size":6479,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26530409,"num_objects":6478,"num_object_clones":0,"num_object_copies":12956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6478,"num_write_kb":25909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":3071}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'6727","reported_seq":6825,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.849138+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:01.849138+0000","last_peered":"2024-09-16T04:51:01.849138+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:01.849138+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":6727,"log_dups_size":0,"ondisk_log_size":6727,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27522875,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6724,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":3208}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'6628","reported_seq":6755,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.454177+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:02.454177+0000","last_peered":"2024-09-16T04:51:02.454177+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:02.454177+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":6628,"log_dups_size":0,"ondisk_log_size":6628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27131904,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3382,"num_write_kb":13528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3299},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'6731","reported_seq":6866,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.435799+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:02.435799+0000","last_peered":"2024-09-16T04:51:02.435799+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:02.435799+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":6731,"log_dups_size":0,"ondisk_log_size":6731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521641,"num_objects":6720,"num_object_clones":0,"num_object_copies":13440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3500,"num_write_kb":14000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3196},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":431935711,"num_objects":105459,"num_object_clones":0,"num_object_copies":210918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1235,"num_objects_unfound":0,"num_objects_dirty":105459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70693,"num_write_kb":282744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2120,"num_bytes_recovered":8650752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":868929536,"data_stored":868849519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105615,"ondisk_log_size":105615,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519848,"kb_used_data":343768,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93851992,"statfs":{"total":96636764160,"available":96104439808,"internally_reserved":0,"allocated":352018432,"data_stored":351759670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":180285097},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593746,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297224,"kb_used_data":157240,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94074616,"statfs":{"total":96636764160,"available":96332406784,"internally_reserved":0,"allocated":161013760,"data_stored":160685418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517116,"kb_used_data":345260,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93854724,"statfs":{"total":96636764160,"available":96107237376,"internally_reserved":0,"allocated":353546240,"data_stored":353287478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":353890304,"data_stored":353886825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162033664,"data_stored":161960605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":353005568,"data_stored":353002089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:04.598 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:05.084 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 128 20 107039 107019 3.26498 2.00781 0.060286 0.0239184 2024-09-16T04:51:05.431 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:05.468 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-16T04:51:04.312093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":431935730,"num_objects":105461,"num_object_clones":0,"num_object_copies":210922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1235,"num_objects_unfound":0,"num_objects_dirty":105461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70695,"num_write_kb":282746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2121,"num_bytes_recovered":8650771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105618,"ondisk_log_size":105618,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1334188,"kb_used_data":846268,"kb_used_omap":12,"kb_used_meta":487859,"kb_avail":281781332,"statfs":{"total":289910292480,"available":288544083968,"internally_reserved":0,"allocated":866578432,"data_stored":865732566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":499567617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,5,5,12,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28254208,"num_objects":6898,"num_object_clones":0,"num_object_copies":13796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-322,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6898,"num_write_kb":27592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":1318912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009449"},"pg_stats":[{"pgid":"2.f","version":"45'6492","reported_seq":6567,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.428781+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:02.428781+0000","last_peered":"2024-09-16T04:51:02.428781+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:02.428781+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6492,"log_dups_size":0,"ondisk_log_size":6492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26591232,"num_objects":6492,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6492,"num_write_kb":25968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3151},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'6611","reported_seq":6722,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:01.917290+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:01.917290+0000","last_peered":"2024-09-16T04:51:01.917290+0000","last_clean":"2024-09-16T04:51:01.917290+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:01.917290+0000","last_undegraded":"2024-09-16T04:51:01.917290+0000","last_fullsized":"2024-09-16T04:51:01.917290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":6611,"log_dups_size":0,"ondisk_log_size":6611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27054080,"num_objects":6605,"num_object_clones":0,"num_object_copies":13210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3580,"num_write_kb":14320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'6552","reported_seq":6686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.927128+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:01.927128+0000","last_peered":"2024-09-16T04:51:01.927128+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:01.927128+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":6552,"log_dups_size":0,"ondisk_log_size":6552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26820608,"num_objects":6548,"num_object_clones":0,"num_object_copies":13096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3291,"num_write_kb":13164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":3229}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'6616","reported_seq":6679,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:01.858379+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:01.858379+0000","last_peered":"2024-09-16T04:51:01.858379+0000","last_clean":"2024-09-16T04:51:01.858379+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:01.858379+0000","last_undegraded":"2024-09-16T04:51:01.858379+0000","last_fullsized":"2024-09-16T04:51:01.858379+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6616,"log_dups_size":0,"ondisk_log_size":6616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27099136,"num_objects":6616,"num_object_clones":0,"num_object_copies":13232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6616,"num_write_kb":26464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'6676","reported_seq":6890,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:02.299699+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:02.299699+0000","last_peered":"2024-09-16T04:51:02.299699+0000","last_clean":"2024-09-16T04:51:02.299699+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:02.299699+0000","last_undegraded":"2024-09-16T04:51:02.299699+0000","last_fullsized":"2024-09-16T04:51:02.299699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":6676,"log_dups_size":0,"ondisk_log_size":6676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27222016,"num_objects":6646,"num_object_clones":0,"num_object_copies":13292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3708,"num_write_kb":14832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'6684","reported_seq":6813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.411862+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:02.411862+0000","last_peered":"2024-09-16T04:51:02.411862+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:02.411862+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":6684,"log_dups_size":0,"ondisk_log_size":6684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27357184,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3677,"num_write_kb":14708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3223},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'6553","reported_seq":6696,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.465656+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:02.465656+0000","last_peered":"2024-09-16T04:51:02.465656+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:02.465656+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":6553,"log_dups_size":0,"ondisk_log_size":6553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26816512,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3208,"num_write_kb":12832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3148},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'6555","reported_seq":6766,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:02.408635+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:02.408635+0000","last_peered":"2024-09-16T04:51:02.408635+0000","last_clean":"2024-09-16T04:51:02.408635+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:02.408635+0000","last_undegraded":"2024-09-16T04:51:02.408635+0000","last_fullsized":"2024-09-16T04:51:02.408635+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":6555,"log_dups_size":0,"ondisk_log_size":6555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26816512,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3625,"num_write_kb":14500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'6541","reported_seq":6764,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.881759+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:01.881759+0000","last_peered":"2024-09-16T04:51:01.881759+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:01.881759+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":6541,"log_dups_size":0,"ondisk_log_size":6541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26656768,"num_objects":6508,"num_object_clones":0,"num_object_copies":13016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3292,"num_write_kb":13168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":3187}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'6689","reported_seq":6814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.343010+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:02.343010+0000","last_peered":"2024-09-16T04:51:02.343010+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:02.343010+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":6689,"log_dups_size":0,"ondisk_log_size":6689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27365376,"num_objects":6681,"num_object_clones":0,"num_object_copies":13362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3344,"num_write_kb":13376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3216},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'6513","reported_seq":6750,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.849689+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:01.849689+0000","last_peered":"2024-09-16T04:51:01.849689+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:01.849689+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":6513,"log_dups_size":0,"ondisk_log_size":6513,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26551506,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6485,"num_write_kb":25931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":3188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'6568","reported_seq":10943,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:51:01.904889+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:51:01.904889+0000","last_peered":"2024-09-16T04:51:01.904889+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:01.904889+0000","last_undegraded":"2024-09-16T04:51:01.904889+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":6568,"log_dups_size":0,"ondisk_log_size":6568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26877952,"num_objects":6562,"num_object_clones":0,"num_object_copies":13124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1235,"num_objects_unfound":0,"num_objects_dirty":6562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3291,"num_write_kb":13164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2111,"num_bytes_recovered":8638464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1235},{"shards":"2","objects":5327}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6479","reported_seq":6570,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.904885+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:01.904885+0000","last_peered":"2024-09-16T04:51:01.904885+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:01.904885+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6479,"log_dups_size":0,"ondisk_log_size":6479,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26530409,"num_objects":6478,"num_object_clones":0,"num_object_copies":12956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6478,"num_write_kb":25909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":3071}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'6727","reported_seq":6825,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.849138+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:01.849138+0000","last_peered":"2024-09-16T04:51:01.849138+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:01.849138+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":6727,"log_dups_size":0,"ondisk_log_size":6727,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27522875,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6724,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":3208}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'6628","reported_seq":6755,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.454177+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:02.454177+0000","last_peered":"2024-09-16T04:51:02.454177+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:02.454177+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":6628,"log_dups_size":0,"ondisk_log_size":6628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27131904,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3382,"num_write_kb":13528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3299},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'6731","reported_seq":6866,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.435799+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:02.435799+0000","last_peered":"2024-09-16T04:51:02.435799+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:02.435799+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":6731,"log_dups_size":0,"ondisk_log_size":6731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521641,"num_objects":6720,"num_object_clones":0,"num_object_copies":13440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3500,"num_write_kb":14000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3196},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":431935711,"num_objects":105459,"num_object_clones":0,"num_object_copies":210918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1235,"num_objects_unfound":0,"num_objects_dirty":105459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70693,"num_write_kb":282744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2120,"num_bytes_recovered":8650752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":868929536,"data_stored":868849519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105615,"ondisk_log_size":105615,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519848,"kb_used_data":343768,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93851992,"statfs":{"total":96636764160,"available":96104439808,"internally_reserved":0,"allocated":352018432,"data_stored":351759670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":180285097},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593746,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297224,"kb_used_data":157240,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94074616,"statfs":{"total":96636764160,"available":96332406784,"internally_reserved":0,"allocated":161013760,"data_stored":160685418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517116,"kb_used_data":345260,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93854724,"statfs":{"total":96636764160,"available":96107237376,"internally_reserved":0,"allocated":353546240,"data_stored":353287478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":353890304,"data_stored":353886825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162033664,"data_stored":161960605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":353005568,"data_stored":353002089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:51:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'6492", 'reported_seq': 6567, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:02.428781+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:51:02.428781+0000', 'last_peered': '2024-09-16T04:51:02.428781+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:51:02.428781+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 6492, 'log_dups_size': 0, 'ondisk_log_size': 6492, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26591232, 'num_objects': 6492, 'num_object_clones': 0, 'num_object_copies': 12984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6492, 'num_write_kb': 25968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3151}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:51:05.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'6552", 'reported_seq': 6686, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:01.927128+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:51:01.927128+0000', 'last_peered': '2024-09-16T04:51:01.927128+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:51:01.927128+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 6552, 'log_dups_size': 0, 'ondisk_log_size': 6552, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26820608, 'num_objects': 6548, 'num_object_clones': 0, 'num_object_copies': 13096, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6548, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3291, 'num_write_kb': 13164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 3229}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:05.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:51:05.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'6684", 'reported_seq': 6813, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:02.411862+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:51:02.411862+0000', 'last_peered': '2024-09-16T04:51:02.411862+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:51:02.411862+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 6684, 'log_dups_size': 0, 'ondisk_log_size': 6684, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27357184, 'num_objects': 6679, 'num_object_clones': 0, 'num_object_copies': 13358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6679, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3677, 'num_write_kb': 14708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3223}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:05.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:51:05.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'6553", 'reported_seq': 6696, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:02.465656+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:51:02.465656+0000', 'last_peered': '2024-09-16T04:51:02.465656+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:51:02.465656+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 6553, 'log_dups_size': 0, 'ondisk_log_size': 6553, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26816512, 'num_objects': 6547, 'num_object_clones': 0, 'num_object_copies': 13094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6547, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3208, 'num_write_kb': 12832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3148}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:05.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:51:05.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'6541", 'reported_seq': 6764, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:01.881759+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:51:01.881759+0000', 'last_peered': '2024-09-16T04:51:01.881759+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:51:01.881759+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 6541, 'log_dups_size': 0, 'ondisk_log_size': 6541, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26656768, 'num_objects': 6508, 'num_object_clones': 0, 'num_object_copies': 13016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6508, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3292, 'num_write_kb': 13168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 3187}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:05.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:51:05.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'6689", 'reported_seq': 6814, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:02.343010+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:51:02.343010+0000', 'last_peered': '2024-09-16T04:51:02.343010+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:51:02.343010+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 6689, 'log_dups_size': 0, 'ondisk_log_size': 6689, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27365376, 'num_objects': 6681, 'num_object_clones': 0, 'num_object_copies': 13362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6681, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3344, 'num_write_kb': 13376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3216}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:05.470 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:51:05.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'6513", 'reported_seq': 6750, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:01.849689+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:51:01.849689+0000', 'last_peered': '2024-09-16T04:51:01.849689+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:51:01.849689+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 6513, 'log_dups_size': 0, 'ondisk_log_size': 6513, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26551506, 'num_objects': 6484, 'num_object_clones': 0, 'num_object_copies': 12968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6484, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6485, 'num_write_kb': 25931, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 3188}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:05.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:51:05.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'6568", 'reported_seq': 10943, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:51:01.904889+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:51:01.904889+0000', 'last_peered': '2024-09-16T04:51:01.904889+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:51:01.904889+0000', 'last_undegraded': '2024-09-16T04:51:01.904889+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 6568, 'log_dups_size': 0, 'ondisk_log_size': 6568, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26877952, 'num_objects': 6562, 'num_object_clones': 0, 'num_object_copies': 13124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1235, 'num_objects_unfound': 0, 'num_objects_dirty': 6562, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3291, 'num_write_kb': 13164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2111, 'num_bytes_recovered': 8638464, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1235}, {'shards': '2', 'objects': 5327}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:05.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:51:05.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'6479", 'reported_seq': 6570, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:01.904885+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:51:01.904885+0000', 'last_peered': '2024-09-16T04:51:01.904885+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:51:01.904885+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 6479, 'log_dups_size': 0, 'ondisk_log_size': 6479, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26530409, 'num_objects': 6478, 'num_object_clones': 0, 'num_object_copies': 12956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6478, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6478, 'num_write_kb': 25909, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 3071}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:05.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:51:05.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'6727", 'reported_seq': 6825, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:01.849138+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:51:01.849138+0000', 'last_peered': '2024-09-16T04:51:01.849138+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:51:01.849138+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 6727, 'log_dups_size': 0, 'ondisk_log_size': 6727, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27522875, 'num_objects': 6722, 'num_object_clones': 0, 'num_object_copies': 13444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6722, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6724, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 3208}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:05.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:51:05.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'6628", 'reported_seq': 6755, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:02.454177+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:51:02.454177+0000', 'last_peered': '2024-09-16T04:51:02.454177+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:51:02.454177+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 6628, 'log_dups_size': 0, 'ondisk_log_size': 6628, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27131904, 'num_objects': 6624, 'num_object_clones': 0, 'num_object_copies': 13248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3382, 'num_write_kb': 13528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3299}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:05.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:51:05.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'6731", 'reported_seq': 6866, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:02.435799+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:51:02.435799+0000', 'last_peered': '2024-09-16T04:51:02.435799+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:51:02.435799+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 6731, 'log_dups_size': 0, 'ondisk_log_size': 6731, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27521641, 'num_objects': 6720, 'num_object_clones': 0, 'num_object_copies': 13440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6720, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3500, 'num_write_kb': 14000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3196}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:05.472 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:51:06.084 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 129 20 107564 107544 3.25557 2.05078 0.0531815 0.0239877 2024-09-16T04:51:06.721 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:06.721 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/210922 objects degraded (17.726%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":214,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":105461,"data_bytes":431935730,"bytes_used":1402384384,"bytes_avail":288507908096,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":210922,"degraded_ratio":0.17725509904135181,"misplaced_objects":1235,"misplaced_total":210922,"misplaced_ratio":0.0058552450668967679,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":109822,"recovering_keys_per_sec":0,"num_objects_recovered":324,"num_bytes_recovered":1318912,"num_keys_recovered":0,"write_bytes_sec":1868354,"read_op_per_sec":0,"write_op_per_sec":456},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:51:06.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:51:06.796 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:07.084 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 130 20 108092 108072 3.24639 2.0625 0.020099 0.0240579 2024-09-16T04:51:07.821 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:07.821 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:07.914 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-16T04:51:06.312547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":431935730,"num_objects":105461,"num_object_clones":0,"num_object_copies":210922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1235,"num_objects_unfound":0,"num_objects_dirty":105461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70695,"num_write_kb":282746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2121,"num_bytes_recovered":8650771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105618,"ondisk_log_size":105618,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1369516,"kb_used_data":855228,"kb_used_omap":12,"kb_used_meta":514227,"kb_avail":281746004,"statfs":{"total":289910292480,"available":288507908096,"internally_reserved":0,"allocated":875753472,"data_stored":874907606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":526568449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,2,5,8,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22437888,"num_objects":5478,"num_object_clones":0,"num_object_copies":10956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-322,"num_objects_unfound":0,"num_objects_dirty":5478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5478,"num_write_kb":21912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":1318912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009436"},"pg_stats":[{"pgid":"2.f","version":"45'6492","reported_seq":6567,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.428781+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:02.428781+0000","last_peered":"2024-09-16T04:51:02.428781+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:02.428781+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6492,"log_dups_size":0,"ondisk_log_size":6492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26591232,"num_objects":6492,"num_object_clones":0,"num_object_copies":12984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6492,"num_write_kb":25968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3151},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'6611","reported_seq":6722,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:01.917290+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:01.917290+0000","last_peered":"2024-09-16T04:51:01.917290+0000","last_clean":"2024-09-16T04:51:01.917290+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:01.917290+0000","last_undegraded":"2024-09-16T04:51:01.917290+0000","last_fullsized":"2024-09-16T04:51:01.917290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":6611,"log_dups_size":0,"ondisk_log_size":6611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27054080,"num_objects":6605,"num_object_clones":0,"num_object_copies":13210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3580,"num_write_kb":14320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'6552","reported_seq":6686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.927128+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:01.927128+0000","last_peered":"2024-09-16T04:51:01.927128+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:01.927128+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":6552,"log_dups_size":0,"ondisk_log_size":6552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26820608,"num_objects":6548,"num_object_clones":0,"num_object_copies":13096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3291,"num_write_kb":13164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":3229}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'6616","reported_seq":6679,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:01.858379+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:01.858379+0000","last_peered":"2024-09-16T04:51:01.858379+0000","last_clean":"2024-09-16T04:51:01.858379+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:01.858379+0000","last_undegraded":"2024-09-16T04:51:01.858379+0000","last_fullsized":"2024-09-16T04:51:01.858379+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6616,"log_dups_size":0,"ondisk_log_size":6616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27099136,"num_objects":6616,"num_object_clones":0,"num_object_copies":13232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6616,"num_write_kb":26464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'6676","reported_seq":6890,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:02.299699+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:02.299699+0000","last_peered":"2024-09-16T04:51:02.299699+0000","last_clean":"2024-09-16T04:51:02.299699+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:02.299699+0000","last_undegraded":"2024-09-16T04:51:02.299699+0000","last_fullsized":"2024-09-16T04:51:02.299699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":6676,"log_dups_size":0,"ondisk_log_size":6676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27222016,"num_objects":6646,"num_object_clones":0,"num_object_copies":13292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3708,"num_write_kb":14832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'6684","reported_seq":6813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.411862+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:02.411862+0000","last_peered":"2024-09-16T04:51:02.411862+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:02.411862+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":6684,"log_dups_size":0,"ondisk_log_size":6684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27357184,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3677,"num_write_kb":14708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3223},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'6553","reported_seq":6696,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.465656+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:02.465656+0000","last_peered":"2024-09-16T04:51:02.465656+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:02.465656+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":6553,"log_dups_size":0,"ondisk_log_size":6553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26816512,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3208,"num_write_kb":12832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3148},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'6555","reported_seq":6766,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:02.408635+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:02.408635+0000","last_peered":"2024-09-16T04:51:02.408635+0000","last_clean":"2024-09-16T04:51:02.408635+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:02.408635+0000","last_undegraded":"2024-09-16T04:51:02.408635+0000","last_fullsized":"2024-09-16T04:51:02.408635+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":6555,"log_dups_size":0,"ondisk_log_size":6555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26816512,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3625,"num_write_kb":14500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'6541","reported_seq":6764,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.881759+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:01.881759+0000","last_peered":"2024-09-16T04:51:01.881759+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:01.881759+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":6541,"log_dups_size":0,"ondisk_log_size":6541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26656768,"num_objects":6508,"num_object_clones":0,"num_object_copies":13016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3292,"num_write_kb":13168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":3187}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'6689","reported_seq":6814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.343010+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:02.343010+0000","last_peered":"2024-09-16T04:51:02.343010+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:02.343010+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":6689,"log_dups_size":0,"ondisk_log_size":6689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27365376,"num_objects":6681,"num_object_clones":0,"num_object_copies":13362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3344,"num_write_kb":13376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3216},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'6513","reported_seq":6750,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.849689+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:01.849689+0000","last_peered":"2024-09-16T04:51:01.849689+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:01.849689+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":6513,"log_dups_size":0,"ondisk_log_size":6513,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26551506,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6485,"num_write_kb":25931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":3188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'6568","reported_seq":10943,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:51:01.904889+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:51:01.904889+0000","last_peered":"2024-09-16T04:51:01.904889+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:01.904889+0000","last_undegraded":"2024-09-16T04:51:01.904889+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":6568,"log_dups_size":0,"ondisk_log_size":6568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26877952,"num_objects":6562,"num_object_clones":0,"num_object_copies":13124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1235,"num_objects_unfound":0,"num_objects_dirty":6562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3291,"num_write_kb":13164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2111,"num_bytes_recovered":8638464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1235},{"shards":"2","objects":5327}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6479","reported_seq":6570,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.904885+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:01.904885+0000","last_peered":"2024-09-16T04:51:01.904885+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:01.904885+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6479,"log_dups_size":0,"ondisk_log_size":6479,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26530409,"num_objects":6478,"num_object_clones":0,"num_object_copies":12956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6478,"num_write_kb":25909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":3071}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'6727","reported_seq":6825,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:01.849138+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:01.849138+0000","last_peered":"2024-09-16T04:51:01.849138+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:01.849138+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":6727,"log_dups_size":0,"ondisk_log_size":6727,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27522875,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6724,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":3208}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'6628","reported_seq":6755,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.454177+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:02.454177+0000","last_peered":"2024-09-16T04:51:02.454177+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:02.454177+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":6628,"log_dups_size":0,"ondisk_log_size":6628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27131904,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3382,"num_write_kb":13528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3299},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'6731","reported_seq":6866,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:02.435799+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:02.435799+0000","last_peered":"2024-09-16T04:51:02.435799+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:02.435799+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":6731,"log_dups_size":0,"ondisk_log_size":6731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521641,"num_objects":6720,"num_object_clones":0,"num_object_copies":13440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3500,"num_write_kb":14000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3196},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":431935711,"num_objects":105459,"num_object_clones":0,"num_object_copies":210918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1235,"num_objects_unfound":0,"num_objects_dirty":105459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70693,"num_write_kb":282744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2120,"num_bytes_recovered":8650752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":878092288,"data_stored":878012271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105615,"ondisk_log_size":105615,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519848,"kb_used_data":343768,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93851992,"statfs":{"total":96636764160,"available":96104439808,"internally_reserved":0,"allocated":352018432,"data_stored":351759670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":180285097},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593747,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332552,"kb_used_data":166200,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94039288,"statfs":{"total":96636764160,"available":96296230912,"internally_reserved":0,"allocated":170188800,"data_stored":169860458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170323630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517116,"kb_used_data":345260,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93854724,"statfs":{"total":96636764160,"available":96107237376,"internally_reserved":0,"allocated":353546240,"data_stored":353287478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":353890304,"data_stored":353886825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":171196416,"data_stored":171123357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":353005568,"data_stored":353002089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:08.084 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 131 20 108612 108592 3.23711 2.03125 0.0400219 0.024127 2024-09-16T04:51:09.086 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 132 20 109176 109156 3.22925 2.20312 0.028401 0.0241854 2024-09-16T04:51:10.086 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 133 20 109727 109707 3.22115 2.15234 0.0182456 0.024246 2024-09-16T04:51:10.915 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:11.086 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 134 20 110266 110246 3.21282 2.10547 0.0542011 0.024306 2024-09-16T04:51:11.753 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:11.753 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:11.833 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-16T04:51:10.314027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":443011314,"num_objects":108165,"num_object_clones":0,"num_object_copies":216330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1081,"num_objects_unfound":0,"num_objects_dirty":108165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73399,"num_write_kb":293562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2275,"num_bytes_recovered":9281555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108322,"ondisk_log_size":108322,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1397036,"kb_used_data":878588,"kb_used_omap":12,"kb_used_meta":518387,"kb_avail":281718484,"statfs":{"total":289910292480,"available":288479727616,"internally_reserved":0,"allocated":899674112,"data_stored":898828246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":530828289},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,9,3,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22052864,"num_objects":5384,"num_object_clones":0,"num_object_copies":10768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-312,"num_objects_unfound":0,"num_objects_dirty":5384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5384,"num_write_kb":21536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":1277952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004586"},"pg_stats":[{"pgid":"2.f","version":"45'6657","reported_seq":6732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:07.511553+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:07.511553+0000","last_peered":"2024-09-16T04:51:07.511553+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:07.511553+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6657,"log_dups_size":0,"ondisk_log_size":6657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27267072,"num_objects":6657,"num_object_clones":0,"num_object_copies":13314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6657,"num_write_kb":26628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3316},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'6793","reported_seq":6905,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:06.856512+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:06.856512+0000","last_peered":"2024-09-16T04:51:06.856512+0000","last_clean":"2024-09-16T04:51:06.856512+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:06.856512+0000","last_undegraded":"2024-09-16T04:51:06.856512+0000","last_fullsized":"2024-09-16T04:51:06.856512+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":6793,"log_dups_size":0,"ondisk_log_size":6793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27799552,"num_objects":6787,"num_object_clones":0,"num_object_copies":13574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3762,"num_write_kb":15048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'6703","reported_seq":6837,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:06.897675+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:06.897675+0000","last_peered":"2024-09-16T04:51:06.897675+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:06.897675+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":6703,"log_dups_size":0,"ondisk_log_size":6703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27439104,"num_objects":6699,"num_object_clones":0,"num_object_copies":13398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3442,"num_write_kb":13768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":3380}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'6790","reported_seq":6856,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:06.912248+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:06.912248+0000","last_peered":"2024-09-16T04:51:06.912248+0000","last_clean":"2024-09-16T04:51:06.912248+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:06.912248+0000","last_undegraded":"2024-09-16T04:51:06.912248+0000","last_fullsized":"2024-09-16T04:51:06.912248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6790,"log_dups_size":0,"ondisk_log_size":6790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27811840,"num_objects":6790,"num_object_clones":0,"num_object_copies":13580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6790,"num_write_kb":27160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'6827","reported_seq":7041,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:07.453419+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:07.453419+0000","last_peered":"2024-09-16T04:51:07.453419+0000","last_clean":"2024-09-16T04:51:07.453419+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:07.453419+0000","last_undegraded":"2024-09-16T04:51:07.453419+0000","last_fullsized":"2024-09-16T04:51:07.453419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":6827,"log_dups_size":0,"ondisk_log_size":6827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27840512,"num_objects":6797,"num_object_clones":0,"num_object_copies":13594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3859,"num_write_kb":15436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'6851","reported_seq":6978,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:07.475942+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:07.475942+0000","last_peered":"2024-09-16T04:51:07.475942+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:07.475942+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":6851,"log_dups_size":0,"ondisk_log_size":6851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041216,"num_objects":6846,"num_object_clones":0,"num_object_copies":13692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3844,"num_write_kb":15376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3390},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'6719","reported_seq":6861,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:07.470160+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:07.470160+0000","last_peered":"2024-09-16T04:51:07.470160+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:07.470160+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":6719,"log_dups_size":0,"ondisk_log_size":6719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27496448,"num_objects":6713,"num_object_clones":0,"num_object_copies":13426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3374,"num_write_kb":13496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3314},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'6738","reported_seq":6949,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:07.481943+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:07.481943+0000","last_peered":"2024-09-16T04:51:07.481943+0000","last_clean":"2024-09-16T04:51:07.481943+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:07.481943+0000","last_undegraded":"2024-09-16T04:51:07.481943+0000","last_fullsized":"2024-09-16T04:51:07.481943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":6738,"log_dups_size":0,"ondisk_log_size":6738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27566080,"num_objects":6730,"num_object_clones":0,"num_object_copies":13460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3808,"num_write_kb":15232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'6714","reported_seq":6935,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:06.873438+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:06.873438+0000","last_peered":"2024-09-16T04:51:06.873438+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:06.873438+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":6714,"log_dups_size":0,"ondisk_log_size":6714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27365376,"num_objects":6681,"num_object_clones":0,"num_object_copies":13362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3465,"num_write_kb":13860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":3360}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'6861","reported_seq":6986,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:07.443065+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:07.443065+0000","last_peered":"2024-09-16T04:51:07.443065+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:07.443065+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":6861,"log_dups_size":0,"ondisk_log_size":6861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28069888,"num_objects":6853,"num_object_clones":0,"num_object_copies":13706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3516,"num_write_kb":14064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3388},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'6678","reported_seq":6914,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:06.931142+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:06.931142+0000","last_peered":"2024-09-16T04:51:06.931142+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:06.931142+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":6678,"log_dups_size":0,"ondisk_log_size":6678,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27227346,"num_objects":6649,"num_object_clones":0,"num_object_copies":13298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6650,"num_write_kb":26591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":3353}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'6749","reported_seq":11433,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:51:06.919559+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:51:06.919559+0000","last_peered":"2024-09-16T04:51:06.919559+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:06.919559+0000","last_undegraded":"2024-09-16T04:51:06.919559+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":6749,"log_dups_size":0,"ondisk_log_size":6749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27619328,"num_objects":6743,"num_object_clones":0,"num_object_copies":13486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1081,"num_objects_unfound":0,"num_objects_dirty":6743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3472,"num_write_kb":13888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2265,"num_bytes_recovered":9269248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1081},{"shards":"2","objects":5662}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6656","reported_seq":6748,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:06.895358+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:06.895358+0000","last_peered":"2024-09-16T04:51:06.895358+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:06.895358+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6656,"log_dups_size":0,"ondisk_log_size":6656,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27255401,"num_objects":6655,"num_object_clones":0,"num_object_copies":13310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6655,"num_write_kb":26617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":3248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'6920","reported_seq":7017,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:06.940126+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:06.940126+0000","last_peered":"2024-09-16T04:51:06.940126+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:06.940126+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":6920,"log_dups_size":0,"ondisk_log_size":6920,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28313403,"num_objects":6915,"num_object_clones":0,"num_object_copies":13830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6917,"num_write_kb":27652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":3401}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'6795","reported_seq":6922,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:07.524272+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:07.524272+0000","last_peered":"2024-09-16T04:51:07.524272+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:07.524272+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":6795,"log_dups_size":0,"ondisk_log_size":6795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27815936,"num_objects":6791,"num_object_clones":0,"num_object_copies":13582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3549,"num_write_kb":14196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3466},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'6868","reported_seq":7003,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:07.451863+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:07.451863+0000","last_peered":"2024-09-16T04:51:07.451863+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:07.451863+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":6868,"log_dups_size":0,"ondisk_log_size":6868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28082793,"num_objects":6857,"num_object_clones":0,"num_object_copies":13714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3637,"num_write_kb":14548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3333},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":443011295,"num_objects":108163,"num_object_clones":0,"num_object_copies":216326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":1081,"num_objects_unfound":0,"num_objects_dirty":108163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73397,"num_write_kb":293560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2274,"num_bytes_recovered":9281536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":900886528,"data_stored":900806511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108319,"ondisk_log_size":108319,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536364,"kb_used_data":351964,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93835476,"statfs":{"total":96636764160,"available":96087527424,"internally_reserved":0,"allocated":360411136,"data_stored":360152374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":188804777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593748,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328584,"kb_used_data":174712,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94043256,"statfs":{"total":96636764160,"available":96300294144,"internally_reserved":0,"allocated":178905088,"data_stored":178576746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532088,"kb_used_data":351912,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93839752,"statfs":{"total":96636764160,"available":96091906048,"internally_reserved":0,"allocated":360357888,"data_stored":360099126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,2,2,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":360738816,"data_stored":360735337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":180056064,"data_stored":179983005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360091648,"data_stored":360088169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:11.834 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:12.089 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 135 20 110763 110743 3.20341 1.94141 0.0495108 0.0243801 2024-09-16T04:51:12.600 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:12.960 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:12.960 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-16T04:51:12.314485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":448364786,"num_objects":109472,"num_object_clones":0,"num_object_copies":218944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":895,"num_objects_unfound":0,"num_objects_dirty":109472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74706,"num_write_kb":298790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2461,"num_bytes_recovered":10043411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109629,"ondisk_log_size":109629,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1391152,"kb_used_data":885184,"kb_used_omap":12,"kb_used_meta":505907,"kb_avail":281724368,"statfs":{"total":289910292480,"available":288485752832,"internally_reserved":0,"allocated":906428416,"data_stored":905582550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":518048769},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,11,5,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27406336,"num_objects":6691,"num_object_clones":0,"num_object_copies":13382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-498,"num_objects_unfound":0,"num_objects_dirty":6691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6691,"num_write_kb":26764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":498,"num_bytes_recovered":2039808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004528"},"pg_stats":[{"pgid":"2.f","version":"45'6657","reported_seq":6732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:07.511553+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:07.511553+0000","last_peered":"2024-09-16T04:51:07.511553+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:07.511553+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6657,"log_dups_size":0,"ondisk_log_size":6657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27267072,"num_objects":6657,"num_object_clones":0,"num_object_copies":13314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6657,"num_write_kb":26628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3316},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'6943","reported_seq":7055,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:11.925301+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:11.925301+0000","last_peered":"2024-09-16T04:51:11.925301+0000","last_clean":"2024-09-16T04:51:11.925301+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:11.925301+0000","last_undegraded":"2024-09-16T04:51:11.925301+0000","last_fullsized":"2024-09-16T04:51:11.925301+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":6943,"log_dups_size":0,"ondisk_log_size":6943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28413952,"num_objects":6937,"num_object_clones":0,"num_object_copies":13874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3912,"num_write_kb":15648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'6850","reported_seq":6984,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:11.943071+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:11.943071+0000","last_peered":"2024-09-16T04:51:11.943071+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:11.943071+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":6850,"log_dups_size":0,"ondisk_log_size":6850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041216,"num_objects":6846,"num_object_clones":0,"num_object_copies":13692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3589,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":3527}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'6968","reported_seq":7033,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:11.932319+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:11.932319+0000","last_peered":"2024-09-16T04:51:11.932319+0000","last_clean":"2024-09-16T04:51:11.932319+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:11.932319+0000","last_undegraded":"2024-09-16T04:51:11.932319+0000","last_fullsized":"2024-09-16T04:51:11.932319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6968,"log_dups_size":0,"ondisk_log_size":6968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28540928,"num_objects":6968,"num_object_clones":0,"num_object_copies":13936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6968,"num_write_kb":27872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'6827","reported_seq":7041,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:07.453419+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:07.453419+0000","last_peered":"2024-09-16T04:51:07.453419+0000","last_clean":"2024-09-16T04:51:07.453419+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:07.453419+0000","last_undegraded":"2024-09-16T04:51:07.453419+0000","last_fullsized":"2024-09-16T04:51:07.453419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":6827,"log_dups_size":0,"ondisk_log_size":6827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27840512,"num_objects":6797,"num_object_clones":0,"num_object_copies":13594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3859,"num_write_kb":15436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'6851","reported_seq":6978,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:07.475942+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:07.475942+0000","last_peered":"2024-09-16T04:51:07.475942+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:07.475942+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":6851,"log_dups_size":0,"ondisk_log_size":6851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041216,"num_objects":6846,"num_object_clones":0,"num_object_copies":13692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3844,"num_write_kb":15376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3390},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'6719","reported_seq":6861,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:07.470160+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:07.470160+0000","last_peered":"2024-09-16T04:51:07.470160+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:07.470160+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":6719,"log_dups_size":0,"ondisk_log_size":6719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27496448,"num_objects":6713,"num_object_clones":0,"num_object_copies":13426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3374,"num_write_kb":13496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3314},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'6738","reported_seq":6949,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:07.481943+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:07.481943+0000","last_peered":"2024-09-16T04:51:07.481943+0000","last_clean":"2024-09-16T04:51:07.481943+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:07.481943+0000","last_undegraded":"2024-09-16T04:51:07.481943+0000","last_fullsized":"2024-09-16T04:51:07.481943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":6738,"log_dups_size":0,"ondisk_log_size":6738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27566080,"num_objects":6730,"num_object_clones":0,"num_object_copies":13460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3808,"num_write_kb":15232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'6901","reported_seq":7124,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:11.905124+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:11.905124+0000","last_peered":"2024-09-16T04:51:11.905124+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:11.905124+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":6901,"log_dups_size":0,"ondisk_log_size":6901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3652,"num_write_kb":14608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":3547}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'6861","reported_seq":6986,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:07.443065+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:07.443065+0000","last_peered":"2024-09-16T04:51:07.443065+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:07.443065+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":6861,"log_dups_size":0,"ondisk_log_size":6861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28069888,"num_objects":6853,"num_object_clones":0,"num_object_copies":13706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3516,"num_write_kb":14064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3388},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'6846","reported_seq":7083,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:11.884154+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:11.884154+0000","last_peered":"2024-09-16T04:51:11.884154+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:11.884154+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":6846,"log_dups_size":0,"ondisk_log_size":6846,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27915474,"num_objects":6817,"num_object_clones":0,"num_object_copies":13634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6818,"num_write_kb":27263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":3521}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'6906","reported_seq":11962,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:51:11.938035+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:51:11.938035+0000","last_peered":"2024-09-16T04:51:11.938035+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:11.938035+0000","last_undegraded":"2024-09-16T04:51:11.938035+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28262400,"num_objects":6900,"num_object_clones":0,"num_object_copies":13800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":895,"num_objects_unfound":0,"num_objects_dirty":6900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3629,"num_write_kb":14516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2451,"num_bytes_recovered":10031104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":895},{"shards":"2","objects":6005}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6810","reported_seq":6903,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:11.884145+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:11.884145+0000","last_peered":"2024-09-16T04:51:11.884145+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:11.884145+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6810,"log_dups_size":0,"ondisk_log_size":6810,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27886185,"num_objects":6809,"num_object_clones":0,"num_object_copies":13618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6809,"num_write_kb":27233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":3402}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7086","reported_seq":7184,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:11.871155+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:11.871155+0000","last_peered":"2024-09-16T04:51:11.871155+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:11.871155+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7086,"log_dups_size":0,"ondisk_log_size":7086,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28993339,"num_objects":7081,"num_object_clones":0,"num_object_copies":14162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7083,"num_write_kb":28316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":3567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'6795","reported_seq":6922,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:07.524272+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:07.524272+0000","last_peered":"2024-09-16T04:51:07.524272+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:07.524272+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":6795,"log_dups_size":0,"ondisk_log_size":6795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27815936,"num_objects":6791,"num_object_clones":0,"num_object_copies":13582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3549,"num_write_kb":14196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3466},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'6868","reported_seq":7003,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:07.451863+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:07.451863+0000","last_peered":"2024-09-16T04:51:07.451863+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:07.451863+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":6868,"log_dups_size":0,"ondisk_log_size":6868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28082793,"num_objects":6857,"num_object_clones":0,"num_object_copies":13714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3637,"num_write_kb":14548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3333},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":448364767,"num_objects":109470,"num_object_clones":0,"num_object_copies":218940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":895,"num_objects_unfound":0,"num_objects_dirty":109470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74704,"num_write_kb":298788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2460,"num_bytes_recovered":10043392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":907677696,"data_stored":907597679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109626,"ondisk_log_size":109626,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530480,"kb_used_data":358560,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93841360,"statfs":{"total":96636764160,"available":96093552640,"internally_reserved":0,"allocated":367165440,"data_stored":366906678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":176025257},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593748,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328584,"kb_used_data":174712,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94043256,"statfs":{"total":96636764160,"available":96300294144,"internally_reserved":0,"allocated":178905088,"data_stored":178576746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532088,"kb_used_data":351912,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93839752,"statfs":{"total":96636764160,"available":96091906048,"internally_reserved":0,"allocated":360357888,"data_stored":360099126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,2,2,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":360738816,"data_stored":360735337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":180056064,"data_stored":179983005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":366882816,"data_stored":366879337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:12.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:51:12.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'6657", 'reported_seq': 6732, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:07.511553+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:51:07.511553+0000', 'last_peered': '2024-09-16T04:51:07.511553+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:51:07.511553+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 6657, 'log_dups_size': 0, 'ondisk_log_size': 6657, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27267072, 'num_objects': 6657, 'num_object_clones': 0, 'num_object_copies': 13314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6657, 'num_write_kb': 26628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3316}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:12.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:51:12.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'6850", 'reported_seq': 6984, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:11.943071+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:51:11.943071+0000', 'last_peered': '2024-09-16T04:51:11.943071+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:51:11.943071+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 6850, 'log_dups_size': 0, 'ondisk_log_size': 6850, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28041216, 'num_objects': 6846, 'num_object_clones': 0, 'num_object_copies': 13692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6846, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3589, 'num_write_kb': 14356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 3527}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:12.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:51:12.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'6851", 'reported_seq': 6978, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:07.475942+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:51:07.475942+0000', 'last_peered': '2024-09-16T04:51:07.475942+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:51:07.475942+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 6851, 'log_dups_size': 0, 'ondisk_log_size': 6851, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28041216, 'num_objects': 6846, 'num_object_clones': 0, 'num_object_copies': 13692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6846, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3844, 'num_write_kb': 15376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3390}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:12.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:51:12.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'6719", 'reported_seq': 6861, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:07.470160+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:51:07.470160+0000', 'last_peered': '2024-09-16T04:51:07.470160+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:51:07.470160+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 6719, 'log_dups_size': 0, 'ondisk_log_size': 6719, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27496448, 'num_objects': 6713, 'num_object_clones': 0, 'num_object_copies': 13426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6713, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3374, 'num_write_kb': 13496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3314}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:12.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:51:12.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'6901", 'reported_seq': 7124, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:11.905124+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:51:11.905124+0000', 'last_peered': '2024-09-16T04:51:11.905124+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:51:11.905124+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 6901, 'log_dups_size': 0, 'ondisk_log_size': 6901, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28131328, 'num_objects': 6868, 'num_object_clones': 0, 'num_object_copies': 13736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3652, 'num_write_kb': 14608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 3547}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:12.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:51:12.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'6861", 'reported_seq': 6986, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:07.443065+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:51:07.443065+0000', 'last_peered': '2024-09-16T04:51:07.443065+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:51:07.443065+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 6861, 'log_dups_size': 0, 'ondisk_log_size': 6861, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28069888, 'num_objects': 6853, 'num_object_clones': 0, 'num_object_copies': 13706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3516, 'num_write_kb': 14064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3388}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:12.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:51:12.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'6846", 'reported_seq': 7083, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:11.884154+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:51:11.884154+0000', 'last_peered': '2024-09-16T04:51:11.884154+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:51:11.884154+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 6846, 'log_dups_size': 0, 'ondisk_log_size': 6846, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27915474, 'num_objects': 6817, 'num_object_clones': 0, 'num_object_copies': 13634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6817, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6818, 'num_write_kb': 27263, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 3521}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:12.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:51:12.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'6906", 'reported_seq': 11962, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:51:11.938035+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:51:11.938035+0000', 'last_peered': '2024-09-16T04:51:11.938035+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:51:11.938035+0000', 'last_undegraded': '2024-09-16T04:51:11.938035+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 6906, 'log_dups_size': 0, 'ondisk_log_size': 6906, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28262400, 'num_objects': 6900, 'num_object_clones': 0, 'num_object_copies': 13800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 895, 'num_objects_unfound': 0, 'num_objects_dirty': 6900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3629, 'num_write_kb': 14516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2451, 'num_bytes_recovered': 10031104, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 895}, {'shards': '2', 'objects': 6005}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:12.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:51:12.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'6810", 'reported_seq': 6903, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:11.884145+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:51:11.884145+0000', 'last_peered': '2024-09-16T04:51:11.884145+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:51:11.884145+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 6810, 'log_dups_size': 0, 'ondisk_log_size': 6810, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27886185, 'num_objects': 6809, 'num_object_clones': 0, 'num_object_copies': 13618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6809, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6809, 'num_write_kb': 27233, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 3402}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:12.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:51:12.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'7086", 'reported_seq': 7184, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:11.871155+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:51:11.871155+0000', 'last_peered': '2024-09-16T04:51:11.871155+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:51:11.871155+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 7086, 'log_dups_size': 0, 'ondisk_log_size': 7086, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28993339, 'num_objects': 7081, 'num_object_clones': 0, 'num_object_copies': 14162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7081, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7083, 'num_write_kb': 28316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 3567}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:12.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:51:12.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'6795", 'reported_seq': 6922, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:07.524272+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:51:07.524272+0000', 'last_peered': '2024-09-16T04:51:07.524272+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:51:07.524272+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 6795, 'log_dups_size': 0, 'ondisk_log_size': 6795, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27815936, 'num_objects': 6791, 'num_object_clones': 0, 'num_object_copies': 13582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6791, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3549, 'num_write_kb': 14196, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3466}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:12.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:51:12.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'6868", 'reported_seq': 7003, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:07.451863+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:51:07.451863+0000', 'last_peered': '2024-09-16T04:51:07.451863+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:51:07.451863+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 6868, 'log_dups_size': 0, 'ondisk_log_size': 6868, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28082793, 'num_objects': 6857, 'num_object_clones': 0, 'num_object_copies': 13714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6857, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3637, 'num_write_kb': 14548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3333}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:12.964 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:51:13.086 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 136 20 111271 111251 3.19444 1.98438 0.135764 0.024447 2024-09-16T04:51:13.945 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:13.945 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/218944 objects degraded (17.076%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":221,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":109472,"data_bytes":448364786,"bytes_used":1424539648,"bytes_avail":288485752832,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":218944,"degraded_ratio":0.17076055977784274,"misplaced_objects":895,"misplaced_total":218944,"misplaced_ratio":0.0040878032738965219,"recovering_objects_per_sec":41,"recovering_bytes_per_sec":169919,"recovering_keys_per_sec":0,"num_objects_recovered":498,"num_bytes_recovered":2039808,"num_keys_recovered":0,"write_bytes_sec":2282999,"read_op_per_sec":0,"write_op_per_sec":557},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:51:13.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:51:13.985 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:14.086 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 137 20 111768 111748 3.1853 1.94141 0.0169593 0.0245176 2024-09-16T04:51:14.975 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:14.975 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:15.016 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-16T04:51:14.314964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":453968114,"num_objects":110840,"num_object_clones":0,"num_object_copies":221680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":895,"num_objects_unfound":0,"num_objects_dirty":110840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76074,"num_write_kb":304262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2461,"num_bytes_recovered":10043411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110997,"ondisk_log_size":110997,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1401884,"kb_used_data":891756,"kb_used_omap":12,"kb_used_meta":510067,"kb_avail":281713636,"statfs":{"total":289910292480,"available":288474763264,"internally_reserved":0,"allocated":913158144,"data_stored":912312278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":522308609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,11,6,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27635712,"num_objects":6747,"num_object_clones":0,"num_object_copies":13494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-340,"num_objects_unfound":0,"num_objects_dirty":6747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6747,"num_write_kb":26988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":1392640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004520"},"pg_stats":[{"pgid":"2.f","version":"45'6834","reported_seq":6909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:12.529686+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:12.529686+0000","last_peered":"2024-09-16T04:51:12.529686+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:12.529686+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6834,"log_dups_size":0,"ondisk_log_size":6834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27992064,"num_objects":6834,"num_object_clones":0,"num_object_copies":13668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6834,"num_write_kb":27336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3493},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'6943","reported_seq":7055,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:11.925301+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:11.925301+0000","last_peered":"2024-09-16T04:51:11.925301+0000","last_clean":"2024-09-16T04:51:11.925301+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:11.925301+0000","last_undegraded":"2024-09-16T04:51:11.925301+0000","last_fullsized":"2024-09-16T04:51:11.925301+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":6943,"log_dups_size":0,"ondisk_log_size":6943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28413952,"num_objects":6937,"num_object_clones":0,"num_object_copies":13874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3912,"num_write_kb":15648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'6850","reported_seq":6984,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:11.943071+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:11.943071+0000","last_peered":"2024-09-16T04:51:11.943071+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:11.943071+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":6850,"log_dups_size":0,"ondisk_log_size":6850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041216,"num_objects":6846,"num_object_clones":0,"num_object_copies":13692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3589,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":3527}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'6968","reported_seq":7033,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:11.932319+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:11.932319+0000","last_peered":"2024-09-16T04:51:11.932319+0000","last_clean":"2024-09-16T04:51:11.932319+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:11.932319+0000","last_undegraded":"2024-09-16T04:51:11.932319+0000","last_fullsized":"2024-09-16T04:51:11.932319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6968,"log_dups_size":0,"ondisk_log_size":6968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28540928,"num_objects":6968,"num_object_clones":0,"num_object_copies":13936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6968,"num_write_kb":27872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'6982","reported_seq":7196,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:12.402972+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:12.402972+0000","last_peered":"2024-09-16T04:51:12.402972+0000","last_clean":"2024-09-16T04:51:12.402972+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:12.402972+0000","last_undegraded":"2024-09-16T04:51:12.402972+0000","last_fullsized":"2024-09-16T04:51:12.402972+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":6982,"log_dups_size":0,"ondisk_log_size":6982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28475392,"num_objects":6952,"num_object_clones":0,"num_object_copies":13904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4014,"num_write_kb":16056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7031","reported_seq":7159,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:12.476240+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:12.476240+0000","last_peered":"2024-09-16T04:51:12.476240+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:12.476240+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7031,"log_dups_size":0,"ondisk_log_size":7031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28778496,"num_objects":7026,"num_object_clones":0,"num_object_copies":14052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4024,"num_write_kb":16096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3570},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'6881","reported_seq":7025,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:12.492269+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:12.492269+0000","last_peered":"2024-09-16T04:51:12.492269+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:12.492269+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":6881,"log_dups_size":0,"ondisk_log_size":6881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28160000,"num_objects":6875,"num_object_clones":0,"num_object_copies":13750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3536,"num_write_kb":14144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3476},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'6921","reported_seq":7132,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:12.517054+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:12.517054+0000","last_peered":"2024-09-16T04:51:12.517054+0000","last_clean":"2024-09-16T04:51:12.517054+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:12.517054+0000","last_undegraded":"2024-09-16T04:51:12.517054+0000","last_fullsized":"2024-09-16T04:51:12.517054+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":6921,"log_dups_size":0,"ondisk_log_size":6921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28315648,"num_objects":6913,"num_object_clones":0,"num_object_copies":13826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3991,"num_write_kb":15964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'6901","reported_seq":7124,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:11.905124+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:11.905124+0000","last_peered":"2024-09-16T04:51:11.905124+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:11.905124+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":6901,"log_dups_size":0,"ondisk_log_size":6901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3652,"num_write_kb":14608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":3547}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7030","reported_seq":7155,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:12.543365+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:12.543365+0000","last_peered":"2024-09-16T04:51:12.543365+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:12.543365+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7030,"log_dups_size":0,"ondisk_log_size":7030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762112,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3685,"num_write_kb":14740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3557},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'6846","reported_seq":7083,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:11.884154+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:11.884154+0000","last_peered":"2024-09-16T04:51:11.884154+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:11.884154+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":6846,"log_dups_size":0,"ondisk_log_size":6846,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27915474,"num_objects":6817,"num_object_clones":0,"num_object_copies":13634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6818,"num_write_kb":27263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":3521}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'6906","reported_seq":11962,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:51:11.938035+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:51:11.938035+0000","last_peered":"2024-09-16T04:51:11.938035+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:11.938035+0000","last_undegraded":"2024-09-16T04:51:11.938035+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28262400,"num_objects":6900,"num_object_clones":0,"num_object_copies":13800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":895,"num_objects_unfound":0,"num_objects_dirty":6900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3629,"num_write_kb":14516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2451,"num_bytes_recovered":10031104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":895},{"shards":"2","objects":6005}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6810","reported_seq":6903,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:11.884145+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:11.884145+0000","last_peered":"2024-09-16T04:51:11.884145+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:11.884145+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6810,"log_dups_size":0,"ondisk_log_size":6810,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27886185,"num_objects":6809,"num_object_clones":0,"num_object_copies":13618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6809,"num_write_kb":27233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":3402}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7086","reported_seq":7184,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:11.871155+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:11.871155+0000","last_peered":"2024-09-16T04:51:11.871155+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:11.871155+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7086,"log_dups_size":0,"ondisk_log_size":7086,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28993339,"num_objects":7081,"num_object_clones":0,"num_object_copies":14162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7083,"num_write_kb":28316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":3567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'6959","reported_seq":7086,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:12.448040+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:12.448040+0000","last_peered":"2024-09-16T04:51:12.448040+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:12.448040+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":6959,"log_dups_size":0,"ondisk_log_size":6959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28487680,"num_objects":6955,"num_object_clones":0,"num_object_copies":13910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3713,"num_write_kb":14852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3630},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7046","reported_seq":7180,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:12.548383+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:12.548383+0000","last_peered":"2024-09-16T04:51:12.548383+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:12.548383+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28811881,"num_objects":7035,"num_object_clones":0,"num_object_copies":14070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3815,"num_write_kb":15260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3511},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":453968095,"num_objects":110838,"num_object_clones":0,"num_object_copies":221676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":895,"num_objects_unfound":0,"num_objects_dirty":110838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":76072,"num_write_kb":304260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2460,"num_bytes_recovered":10043392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":914583552,"data_stored":914503535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110994,"ondisk_log_size":110994,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530480,"kb_used_data":358560,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93841360,"statfs":{"total":96636764160,"available":96093552640,"internally_reserved":0,"allocated":367165440,"data_stored":366906678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":176025257},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593748,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328584,"kb_used_data":174712,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94043256,"statfs":{"total":96636764160,"available":96300294144,"internally_reserved":0,"allocated":178905088,"data_stored":178576746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542820,"kb_used_data":358484,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93829020,"statfs":{"total":96636764160,"available":96080916480,"internally_reserved":0,"allocated":367087616,"data_stored":366828854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,3,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":367644672,"data_stored":367641193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":180056064,"data_stored":179983005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":366882816,"data_stored":366879337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:15.086 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 138 20 112277 112257 3.17663 1.98828 0.04773 0.0245853 2024-09-16T04:51:16.086 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 139 20 112822 112802 3.16909 2.12891 0.119907 0.0246413 2024-09-16T04:51:17.087 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:2024-09-16T04:51:17.085998+0000 min lat: 0.00206892 max lat: 2.69414 avg lat: 0.0247018 2024-09-16T04:51:17.087 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T04:51:17.087 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 140 20 113358 113338 3.16141 2.09375 0.0243753 0.0247018 2024-09-16T04:51:18.019 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:18.087 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 141 20 113881 113861 3.15348 2.04297 0.0262948 0.0247642 2024-09-16T04:51:18.919 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:18.921 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:18.973 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-16T04:51:18.315904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":464666866,"num_objects":113452,"num_object_clones":0,"num_object_copies":226904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":701,"num_objects_unfound":0,"num_objects_dirty":113452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78686,"num_write_kb":314710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2655,"num_bytes_recovered":10838035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113609,"ondisk_log_size":113609,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1458144,"kb_used_data":911920,"kb_used_omap":12,"kb_used_meta":546163,"kb_avail":281657376,"statfs":{"total":289910292480,"available":288417153024,"internally_reserved":0,"allocated":933806080,"data_stored":932960214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":559270913},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,5,3,8,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32731136,"num_objects":7991,"num_object_clones":0,"num_object_copies":15982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-534,"num_objects_unfound":0,"num_objects_dirty":7991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7991,"num_write_kb":31964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":2187264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003356"},"pg_stats":[{"pgid":"2.f","version":"45'7005","reported_seq":7080,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.623999+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:17.623999+0000","last_peered":"2024-09-16T04:51:17.623999+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:17.623999+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7005,"log_dups_size":0,"ondisk_log_size":7005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28692480,"num_objects":7005,"num_object_clones":0,"num_object_copies":14010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7005,"num_write_kb":28020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3664},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7097","reported_seq":7209,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:16.936763+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:16.936763+0000","last_peered":"2024-09-16T04:51:16.936763+0000","last_clean":"2024-09-16T04:51:16.936763+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:16.936763+0000","last_undegraded":"2024-09-16T04:51:16.936763+0000","last_fullsized":"2024-09-16T04:51:16.936763+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7097,"log_dups_size":0,"ondisk_log_size":7097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29044736,"num_objects":7091,"num_object_clones":0,"num_object_copies":14182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4066,"num_write_kb":16264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7034","reported_seq":7168,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.945483+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:16.945483+0000","last_peered":"2024-09-16T04:51:16.945483+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:16.945483+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7034,"log_dups_size":0,"ondisk_log_size":7034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28794880,"num_objects":7030,"num_object_clones":0,"num_object_copies":14060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3773,"num_write_kb":15092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":3711}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7125","reported_seq":7191,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:16.929361+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:16.929361+0000","last_peered":"2024-09-16T04:51:16.929361+0000","last_clean":"2024-09-16T04:51:16.929361+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:16.929361+0000","last_undegraded":"2024-09-16T04:51:16.929361+0000","last_fullsized":"2024-09-16T04:51:16.929361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7125,"log_dups_size":0,"ondisk_log_size":7125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29184000,"num_objects":7125,"num_object_clones":0,"num_object_copies":14250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7125,"num_write_kb":28500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7134","reported_seq":7348,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:17.547442+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:17.547442+0000","last_peered":"2024-09-16T04:51:17.547442+0000","last_clean":"2024-09-16T04:51:17.547442+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:17.547442+0000","last_undegraded":"2024-09-16T04:51:17.547442+0000","last_fullsized":"2024-09-16T04:51:17.547442+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7134,"log_dups_size":0,"ondisk_log_size":7134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29097984,"num_objects":7104,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4166,"num_write_kb":16664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7177","reported_seq":7305,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.584449+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:17.584449+0000","last_peered":"2024-09-16T04:51:17.584449+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:17.584449+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7177,"log_dups_size":0,"ondisk_log_size":7177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29376512,"num_objects":7172,"num_object_clones":0,"num_object_copies":14344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4170,"num_write_kb":16680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3716},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7047","reported_seq":7191,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.543052+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:17.543052+0000","last_peered":"2024-09-16T04:51:17.543052+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:17.543052+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7047,"log_dups_size":0,"ondisk_log_size":7047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28839936,"num_objects":7041,"num_object_clones":0,"num_object_copies":14082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3702,"num_write_kb":14808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3642},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7091","reported_seq":7301,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:17.526662+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:17.526662+0000","last_peered":"2024-09-16T04:51:17.526662+0000","last_clean":"2024-09-16T04:51:17.526662+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:17.526662+0000","last_undegraded":"2024-09-16T04:51:17.526662+0000","last_fullsized":"2024-09-16T04:51:17.526662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7091,"log_dups_size":0,"ondisk_log_size":7091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29011968,"num_objects":7083,"num_object_clones":0,"num_object_copies":14166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4161,"num_write_kb":16644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7071","reported_seq":7294,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.916537+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:16.916537+0000","last_peered":"2024-09-16T04:51:16.916537+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:16.916537+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7071,"log_dups_size":0,"ondisk_log_size":7071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28827648,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3822,"num_write_kb":15288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":3717}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7182","reported_seq":7307,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.580858+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:17.580858+0000","last_peered":"2024-09-16T04:51:17.580858+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:17.580858+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7182,"log_dups_size":0,"ondisk_log_size":7182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29384704,"num_objects":7174,"num_object_clones":0,"num_object_copies":14348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3837,"num_write_kb":15348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3709},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7011","reported_seq":7248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.828901+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:16.828901+0000","last_peered":"2024-09-16T04:51:16.828901+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:16.828901+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7011,"log_dups_size":0,"ondisk_log_size":7011,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28591314,"num_objects":6982,"num_object_clones":0,"num_object_copies":13964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6983,"num_write_kb":27923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":3686}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7058","reported_seq":12502,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:51:16.955155+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:51:16.955155+0000","last_peered":"2024-09-16T04:51:16.955155+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:16.955155+0000","last_undegraded":"2024-09-16T04:51:16.955155+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7058,"log_dups_size":0,"ondisk_log_size":7058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28884992,"num_objects":7052,"num_object_clones":0,"num_object_copies":14104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":701,"num_objects_unfound":0,"num_objects_dirty":7052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3781,"num_write_kb":15124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2645,"num_bytes_recovered":10825728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":701},{"shards":"2","objects":6351}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6979","reported_seq":7072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.928770+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:16.928770+0000","last_peered":"2024-09-16T04:51:16.928770+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:16.928770+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6979,"log_dups_size":0,"ondisk_log_size":6979,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28578409,"num_objects":6978,"num_object_clones":0,"num_object_copies":13956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6978,"num_write_kb":27909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":3571}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7246","reported_seq":7344,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.899521+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:16.899521+0000","last_peered":"2024-09-16T04:51:16.899521+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:16.899521+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7246,"log_dups_size":0,"ondisk_log_size":7246,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29648699,"num_objects":7241,"num_object_clones":0,"num_object_copies":14482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7243,"num_write_kb":28956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":3727}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7134","reported_seq":7261,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.517108+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:17.517108+0000","last_peered":"2024-09-16T04:51:17.517108+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:17.517108+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7134,"log_dups_size":0,"ondisk_log_size":7134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29204480,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3888,"num_write_kb":15552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3805},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7215","reported_seq":7349,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.564989+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:17.564989+0000","last_peered":"2024-09-16T04:51:17.564989+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:17.564989+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":7215,"log_dups_size":0,"ondisk_log_size":7215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29504105,"num_objects":7204,"num_object_clones":0,"num_object_copies":14408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3984,"num_write_kb":15936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3680},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":464666847,"num_objects":113450,"num_object_clones":0,"num_object_copies":226900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":701,"num_objects_unfound":0,"num_objects_dirty":113450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78684,"num_write_kb":314708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2654,"num_bytes_recovered":10838016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":936808448,"data_stored":936728431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113606,"ondisk_log_size":113606,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557860,"kb_used_data":363732,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93813980,"statfs":{"total":96636764160,"available":96065515520,"internally_reserved":0,"allocated":372461568,"data_stored":372202806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":198766249},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593749,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":345560,"kb_used_data":183368,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94026280,"statfs":{"total":96636764160,"available":96282910720,"internally_reserved":0,"allocated":187768832,"data_stored":187440490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":166063790},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554724,"kb_used_data":364820,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93817116,"statfs":{"total":96636764160,"available":96068726784,"internally_reserved":0,"allocated":373575680,"data_stored":373316918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":194440874},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,2,5,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":374329344,"data_stored":374325865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188928000,"data_stored":188854941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":373551104,"data_stored":373547625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:18.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:19.087 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 142 20 114388 114368 3.14522 1.98047 0.0704427 0.0248301 2024-09-16T04:51:19.911 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:19.912 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:19.961 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-16T04:51:18.315904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":464666866,"num_objects":113452,"num_object_clones":0,"num_object_copies":226904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":701,"num_objects_unfound":0,"num_objects_dirty":113452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78686,"num_write_kb":314710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2655,"num_bytes_recovered":10838035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113609,"ondisk_log_size":113609,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1458144,"kb_used_data":911920,"kb_used_omap":12,"kb_used_meta":546163,"kb_avail":281657376,"statfs":{"total":289910292480,"available":288417153024,"internally_reserved":0,"allocated":933806080,"data_stored":932960214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":559270913},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,5,3,8,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32731136,"num_objects":7991,"num_object_clones":0,"num_object_copies":15982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-534,"num_objects_unfound":0,"num_objects_dirty":7991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7991,"num_write_kb":31964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":2187264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003356"},"pg_stats":[{"pgid":"2.f","version":"45'7005","reported_seq":7080,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.623999+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:17.623999+0000","last_peered":"2024-09-16T04:51:17.623999+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:17.623999+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7005,"log_dups_size":0,"ondisk_log_size":7005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28692480,"num_objects":7005,"num_object_clones":0,"num_object_copies":14010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7005,"num_write_kb":28020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3664},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7097","reported_seq":7209,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:16.936763+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:16.936763+0000","last_peered":"2024-09-16T04:51:16.936763+0000","last_clean":"2024-09-16T04:51:16.936763+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:16.936763+0000","last_undegraded":"2024-09-16T04:51:16.936763+0000","last_fullsized":"2024-09-16T04:51:16.936763+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7097,"log_dups_size":0,"ondisk_log_size":7097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29044736,"num_objects":7091,"num_object_clones":0,"num_object_copies":14182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4066,"num_write_kb":16264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7034","reported_seq":7168,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.945483+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:16.945483+0000","last_peered":"2024-09-16T04:51:16.945483+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:16.945483+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7034,"log_dups_size":0,"ondisk_log_size":7034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28794880,"num_objects":7030,"num_object_clones":0,"num_object_copies":14060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3773,"num_write_kb":15092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":3711}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7125","reported_seq":7191,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:16.929361+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:16.929361+0000","last_peered":"2024-09-16T04:51:16.929361+0000","last_clean":"2024-09-16T04:51:16.929361+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:16.929361+0000","last_undegraded":"2024-09-16T04:51:16.929361+0000","last_fullsized":"2024-09-16T04:51:16.929361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7125,"log_dups_size":0,"ondisk_log_size":7125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29184000,"num_objects":7125,"num_object_clones":0,"num_object_copies":14250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7125,"num_write_kb":28500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7134","reported_seq":7348,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:17.547442+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:17.547442+0000","last_peered":"2024-09-16T04:51:17.547442+0000","last_clean":"2024-09-16T04:51:17.547442+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:17.547442+0000","last_undegraded":"2024-09-16T04:51:17.547442+0000","last_fullsized":"2024-09-16T04:51:17.547442+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7134,"log_dups_size":0,"ondisk_log_size":7134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29097984,"num_objects":7104,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4166,"num_write_kb":16664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7177","reported_seq":7305,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.584449+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:17.584449+0000","last_peered":"2024-09-16T04:51:17.584449+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:17.584449+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7177,"log_dups_size":0,"ondisk_log_size":7177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29376512,"num_objects":7172,"num_object_clones":0,"num_object_copies":14344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4170,"num_write_kb":16680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3716},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7047","reported_seq":7191,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.543052+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:17.543052+0000","last_peered":"2024-09-16T04:51:17.543052+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:17.543052+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7047,"log_dups_size":0,"ondisk_log_size":7047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28839936,"num_objects":7041,"num_object_clones":0,"num_object_copies":14082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3702,"num_write_kb":14808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3642},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7091","reported_seq":7301,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:17.526662+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:17.526662+0000","last_peered":"2024-09-16T04:51:17.526662+0000","last_clean":"2024-09-16T04:51:17.526662+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:17.526662+0000","last_undegraded":"2024-09-16T04:51:17.526662+0000","last_fullsized":"2024-09-16T04:51:17.526662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7091,"log_dups_size":0,"ondisk_log_size":7091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29011968,"num_objects":7083,"num_object_clones":0,"num_object_copies":14166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4161,"num_write_kb":16644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7071","reported_seq":7294,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.916537+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:16.916537+0000","last_peered":"2024-09-16T04:51:16.916537+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:16.916537+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7071,"log_dups_size":0,"ondisk_log_size":7071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28827648,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3822,"num_write_kb":15288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":3717}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7182","reported_seq":7307,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.580858+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:17.580858+0000","last_peered":"2024-09-16T04:51:17.580858+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:17.580858+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7182,"log_dups_size":0,"ondisk_log_size":7182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29384704,"num_objects":7174,"num_object_clones":0,"num_object_copies":14348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3837,"num_write_kb":15348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3709},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7011","reported_seq":7248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.828901+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:16.828901+0000","last_peered":"2024-09-16T04:51:16.828901+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:16.828901+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7011,"log_dups_size":0,"ondisk_log_size":7011,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28591314,"num_objects":6982,"num_object_clones":0,"num_object_copies":13964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6983,"num_write_kb":27923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":3686}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7058","reported_seq":12502,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:51:16.955155+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:51:16.955155+0000","last_peered":"2024-09-16T04:51:16.955155+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:16.955155+0000","last_undegraded":"2024-09-16T04:51:16.955155+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7058,"log_dups_size":0,"ondisk_log_size":7058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28884992,"num_objects":7052,"num_object_clones":0,"num_object_copies":14104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":701,"num_objects_unfound":0,"num_objects_dirty":7052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3781,"num_write_kb":15124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2645,"num_bytes_recovered":10825728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":701},{"shards":"2","objects":6351}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6979","reported_seq":7072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.928770+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:16.928770+0000","last_peered":"2024-09-16T04:51:16.928770+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:16.928770+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6979,"log_dups_size":0,"ondisk_log_size":6979,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28578409,"num_objects":6978,"num_object_clones":0,"num_object_copies":13956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6978,"num_write_kb":27909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":3571}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7246","reported_seq":7344,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.899521+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:16.899521+0000","last_peered":"2024-09-16T04:51:16.899521+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:16.899521+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7246,"log_dups_size":0,"ondisk_log_size":7246,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29648699,"num_objects":7241,"num_object_clones":0,"num_object_copies":14482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7243,"num_write_kb":28956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":3727}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7134","reported_seq":7261,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.517108+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:17.517108+0000","last_peered":"2024-09-16T04:51:17.517108+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:17.517108+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7134,"log_dups_size":0,"ondisk_log_size":7134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29204480,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3888,"num_write_kb":15552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3805},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7215","reported_seq":7349,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.564989+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:17.564989+0000","last_peered":"2024-09-16T04:51:17.564989+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:17.564989+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":7215,"log_dups_size":0,"ondisk_log_size":7215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29504105,"num_objects":7204,"num_object_clones":0,"num_object_copies":14408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3984,"num_write_kb":15936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3680},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":464666847,"num_objects":113450,"num_object_clones":0,"num_object_copies":226900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":701,"num_objects_unfound":0,"num_objects_dirty":113450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78684,"num_write_kb":314708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2654,"num_bytes_recovered":10838016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":936808448,"data_stored":936728431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113606,"ondisk_log_size":113606,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557860,"kb_used_data":363732,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93813980,"statfs":{"total":96636764160,"available":96065515520,"internally_reserved":0,"allocated":372461568,"data_stored":372202806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":198766249},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593749,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":345560,"kb_used_data":183368,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94026280,"statfs":{"total":96636764160,"available":96282910720,"internally_reserved":0,"allocated":187768832,"data_stored":187440490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":166063790},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554724,"kb_used_data":364820,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93817116,"statfs":{"total":96636764160,"available":96068726784,"internally_reserved":0,"allocated":373575680,"data_stored":373316918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":194440874},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,2,5,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":374329344,"data_stored":374325865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188928000,"data_stored":188854941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":373551104,"data_stored":373547625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:19.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:51:19.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'7005", 'reported_seq': 7080, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:17.623999+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:51:17.623999+0000', 'last_peered': '2024-09-16T04:51:17.623999+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:51:17.623999+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 7005, 'log_dups_size': 0, 'ondisk_log_size': 7005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28692480, 'num_objects': 7005, 'num_object_clones': 0, 'num_object_copies': 14010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7005, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7005, 'num_write_kb': 28020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3664}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:19.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:51:19.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'7034", 'reported_seq': 7168, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:16.945483+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:51:16.945483+0000', 'last_peered': '2024-09-16T04:51:16.945483+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:51:16.945483+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 7034, 'log_dups_size': 0, 'ondisk_log_size': 7034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28794880, 'num_objects': 7030, 'num_object_clones': 0, 'num_object_copies': 14060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7030, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3773, 'num_write_kb': 15092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 3711}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:19.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:51:19.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'7177", 'reported_seq': 7305, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:17.584449+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:51:17.584449+0000', 'last_peered': '2024-09-16T04:51:17.584449+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:51:17.584449+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 7177, 'log_dups_size': 0, 'ondisk_log_size': 7177, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29376512, 'num_objects': 7172, 'num_object_clones': 0, 'num_object_copies': 14344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7172, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4170, 'num_write_kb': 16680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3716}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:19.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:51:19.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'7047", 'reported_seq': 7191, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:17.543052+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:51:17.543052+0000', 'last_peered': '2024-09-16T04:51:17.543052+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:51:17.543052+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 7047, 'log_dups_size': 0, 'ondisk_log_size': 7047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28839936, 'num_objects': 7041, 'num_object_clones': 0, 'num_object_copies': 14082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7041, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3702, 'num_write_kb': 14808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3642}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:19.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:51:19.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'7071", 'reported_seq': 7294, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:16.916537+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:51:16.916537+0000', 'last_peered': '2024-09-16T04:51:16.916537+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:51:16.916537+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 7071, 'log_dups_size': 0, 'ondisk_log_size': 7071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28827648, 'num_objects': 7038, 'num_object_clones': 0, 'num_object_copies': 14076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7038, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3822, 'num_write_kb': 15288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 3717}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:19.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:51:19.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'7182", 'reported_seq': 7307, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:17.580858+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:51:17.580858+0000', 'last_peered': '2024-09-16T04:51:17.580858+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:51:17.580858+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 7182, 'log_dups_size': 0, 'ondisk_log_size': 7182, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29384704, 'num_objects': 7174, 'num_object_clones': 0, 'num_object_copies': 14348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7174, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3837, 'num_write_kb': 15348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3709}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:19.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:51:19.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'7011", 'reported_seq': 7248, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:16.828901+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:51:16.828901+0000', 'last_peered': '2024-09-16T04:51:16.828901+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:51:16.828901+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 7011, 'log_dups_size': 0, 'ondisk_log_size': 7011, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28591314, 'num_objects': 6982, 'num_object_clones': 0, 'num_object_copies': 13964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6983, 'num_write_kb': 27923, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 3686}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:19.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:51:19.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'7058", 'reported_seq': 12502, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:51:16.955155+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:51:16.955155+0000', 'last_peered': '2024-09-16T04:51:16.955155+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:51:16.955155+0000', 'last_undegraded': '2024-09-16T04:51:16.955155+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 7058, 'log_dups_size': 0, 'ondisk_log_size': 7058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28884992, 'num_objects': 7052, 'num_object_clones': 0, 'num_object_copies': 14104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 701, 'num_objects_unfound': 0, 'num_objects_dirty': 7052, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3781, 'num_write_kb': 15124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2645, 'num_bytes_recovered': 10825728, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 701}, {'shards': '2', 'objects': 6351}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:19.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:51:19.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'6979", 'reported_seq': 7072, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:16.928770+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:51:16.928770+0000', 'last_peered': '2024-09-16T04:51:16.928770+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:51:16.928770+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 6979, 'log_dups_size': 0, 'ondisk_log_size': 6979, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28578409, 'num_objects': 6978, 'num_object_clones': 0, 'num_object_copies': 13956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6978, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6978, 'num_write_kb': 27909, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 3571}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:19.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:51:19.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'7246", 'reported_seq': 7344, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:16.899521+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:51:16.899521+0000', 'last_peered': '2024-09-16T04:51:16.899521+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:51:16.899521+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 7246, 'log_dups_size': 0, 'ondisk_log_size': 7246, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29648699, 'num_objects': 7241, 'num_object_clones': 0, 'num_object_copies': 14482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7241, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7243, 'num_write_kb': 28956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 3727}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:19.966 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:51:19.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'7134", 'reported_seq': 7261, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:17.517108+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:51:17.517108+0000', 'last_peered': '2024-09-16T04:51:17.517108+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:51:17.517108+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 7134, 'log_dups_size': 0, 'ondisk_log_size': 7134, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29204480, 'num_objects': 7130, 'num_object_clones': 0, 'num_object_copies': 14260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3888, 'num_write_kb': 15552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3805}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:19.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:51:19.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'7215", 'reported_seq': 7349, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:17.564989+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:51:17.564989+0000', 'last_peered': '2024-09-16T04:51:17.564989+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:51:17.564989+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 7215, 'log_dups_size': 0, 'ondisk_log_size': 7215, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29504105, 'num_objects': 7204, 'num_object_clones': 0, 'num_object_copies': 14408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7204, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3984, 'num_write_kb': 15936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3680}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:19.967 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:51:20.087 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 143 20 114895 114875 3.13707 1.98047 0.0393844 0.0248957 2024-09-16T04:51:21.038 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:21.038 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/226904 objects degraded (16.477%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":228,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":113452,"data_bytes":464666866,"bytes_used":1493139456,"bytes_avail":288417153024,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":226904,"degraded_ratio":0.16477012304763247,"misplaced_objects":701,"misplaced_total":226904,"misplaced_ratio":0.0030894122624546059,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":129674,"recovering_keys_per_sec":0,"num_objects_recovered":380,"num_bytes_recovered":1556480,"num_keys_recovered":0,"write_bytes_sec":1804175,"read_op_per_sec":0,"write_op_per_sec":440},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:51:21.087 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 144 20 115387 115367 3.12863 1.92188 0.110929 0.0249626 2024-09-16T04:51:21.092 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:51:21.093 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:21.804 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:21.805 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:21.870 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-09-16T04:51:20.316529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":464666866,"num_objects":113452,"num_object_clones":0,"num_object_copies":226904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":701,"num_objects_unfound":0,"num_objects_dirty":113452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78686,"num_write_kb":314710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2655,"num_bytes_recovered":10838035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113609,"ondisk_log_size":113609,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1458144,"kb_used_data":911920,"kb_used_omap":12,"kb_used_meta":546163,"kb_avail":281657376,"statfs":{"total":289910292480,"available":288417153024,"internally_reserved":0,"allocated":933806080,"data_stored":932960214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":559270913},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,5,3,8,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21655552,"num_objects":5287,"num_object_clones":0,"num_object_copies":10574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-380,"num_objects_unfound":0,"num_objects_dirty":5287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5287,"num_write_kb":21148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":1556480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003021"},"pg_stats":[{"pgid":"2.f","version":"45'7005","reported_seq":7080,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.623999+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:17.623999+0000","last_peered":"2024-09-16T04:51:17.623999+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:17.623999+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7005,"log_dups_size":0,"ondisk_log_size":7005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28692480,"num_objects":7005,"num_object_clones":0,"num_object_copies":14010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7005,"num_write_kb":28020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3664},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7097","reported_seq":7209,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:16.936763+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:16.936763+0000","last_peered":"2024-09-16T04:51:16.936763+0000","last_clean":"2024-09-16T04:51:16.936763+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:16.936763+0000","last_undegraded":"2024-09-16T04:51:16.936763+0000","last_fullsized":"2024-09-16T04:51:16.936763+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7097,"log_dups_size":0,"ondisk_log_size":7097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29044736,"num_objects":7091,"num_object_clones":0,"num_object_copies":14182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4066,"num_write_kb":16264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7034","reported_seq":7168,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.945483+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:16.945483+0000","last_peered":"2024-09-16T04:51:16.945483+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:16.945483+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7034,"log_dups_size":0,"ondisk_log_size":7034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28794880,"num_objects":7030,"num_object_clones":0,"num_object_copies":14060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3773,"num_write_kb":15092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":3711}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7125","reported_seq":7191,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:16.929361+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:16.929361+0000","last_peered":"2024-09-16T04:51:16.929361+0000","last_clean":"2024-09-16T04:51:16.929361+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:16.929361+0000","last_undegraded":"2024-09-16T04:51:16.929361+0000","last_fullsized":"2024-09-16T04:51:16.929361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7125,"log_dups_size":0,"ondisk_log_size":7125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29184000,"num_objects":7125,"num_object_clones":0,"num_object_copies":14250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7125,"num_write_kb":28500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7134","reported_seq":7348,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:17.547442+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:17.547442+0000","last_peered":"2024-09-16T04:51:17.547442+0000","last_clean":"2024-09-16T04:51:17.547442+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:17.547442+0000","last_undegraded":"2024-09-16T04:51:17.547442+0000","last_fullsized":"2024-09-16T04:51:17.547442+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7134,"log_dups_size":0,"ondisk_log_size":7134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29097984,"num_objects":7104,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4166,"num_write_kb":16664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7177","reported_seq":7305,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.584449+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:17.584449+0000","last_peered":"2024-09-16T04:51:17.584449+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:17.584449+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7177,"log_dups_size":0,"ondisk_log_size":7177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29376512,"num_objects":7172,"num_object_clones":0,"num_object_copies":14344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4170,"num_write_kb":16680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3716},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7047","reported_seq":7191,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.543052+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:17.543052+0000","last_peered":"2024-09-16T04:51:17.543052+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:17.543052+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7047,"log_dups_size":0,"ondisk_log_size":7047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28839936,"num_objects":7041,"num_object_clones":0,"num_object_copies":14082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3702,"num_write_kb":14808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3642},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7091","reported_seq":7301,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:17.526662+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:17.526662+0000","last_peered":"2024-09-16T04:51:17.526662+0000","last_clean":"2024-09-16T04:51:17.526662+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:17.526662+0000","last_undegraded":"2024-09-16T04:51:17.526662+0000","last_fullsized":"2024-09-16T04:51:17.526662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7091,"log_dups_size":0,"ondisk_log_size":7091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29011968,"num_objects":7083,"num_object_clones":0,"num_object_copies":14166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4161,"num_write_kb":16644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7071","reported_seq":7294,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.916537+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:16.916537+0000","last_peered":"2024-09-16T04:51:16.916537+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:16.916537+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7071,"log_dups_size":0,"ondisk_log_size":7071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28827648,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3822,"num_write_kb":15288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":3717}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7182","reported_seq":7307,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.580858+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:17.580858+0000","last_peered":"2024-09-16T04:51:17.580858+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:17.580858+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7182,"log_dups_size":0,"ondisk_log_size":7182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29384704,"num_objects":7174,"num_object_clones":0,"num_object_copies":14348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3837,"num_write_kb":15348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3709},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7011","reported_seq":7248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.828901+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:16.828901+0000","last_peered":"2024-09-16T04:51:16.828901+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:16.828901+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7011,"log_dups_size":0,"ondisk_log_size":7011,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28591314,"num_objects":6982,"num_object_clones":0,"num_object_copies":13964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6983,"num_write_kb":27923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":3686}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7058","reported_seq":12502,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:51:16.955155+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:51:16.955155+0000","last_peered":"2024-09-16T04:51:16.955155+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:16.955155+0000","last_undegraded":"2024-09-16T04:51:16.955155+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7058,"log_dups_size":0,"ondisk_log_size":7058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28884992,"num_objects":7052,"num_object_clones":0,"num_object_copies":14104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":701,"num_objects_unfound":0,"num_objects_dirty":7052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3781,"num_write_kb":15124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2645,"num_bytes_recovered":10825728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":701},{"shards":"2","objects":6351}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'6979","reported_seq":7072,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.928770+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:16.928770+0000","last_peered":"2024-09-16T04:51:16.928770+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:16.928770+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":6979,"log_dups_size":0,"ondisk_log_size":6979,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28578409,"num_objects":6978,"num_object_clones":0,"num_object_copies":13956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6978,"num_write_kb":27909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":3571}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7246","reported_seq":7344,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:16.899521+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:16.899521+0000","last_peered":"2024-09-16T04:51:16.899521+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:16.899521+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7246,"log_dups_size":0,"ondisk_log_size":7246,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29648699,"num_objects":7241,"num_object_clones":0,"num_object_copies":14482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7243,"num_write_kb":28956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":3727}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7134","reported_seq":7261,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.517108+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:17.517108+0000","last_peered":"2024-09-16T04:51:17.517108+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:17.517108+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7134,"log_dups_size":0,"ondisk_log_size":7134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29204480,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3888,"num_write_kb":15552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3805},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7215","reported_seq":7349,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:17.564989+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:17.564989+0000","last_peered":"2024-09-16T04:51:17.564989+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:17.564989+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":7215,"log_dups_size":0,"ondisk_log_size":7215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29504105,"num_objects":7204,"num_object_clones":0,"num_object_copies":14408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3984,"num_write_kb":15936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3680},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":464666847,"num_objects":113450,"num_object_clones":0,"num_object_copies":226900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":701,"num_objects_unfound":0,"num_objects_dirty":113450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78684,"num_write_kb":314708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2654,"num_bytes_recovered":10838016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":936808448,"data_stored":936728431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113606,"ondisk_log_size":113606,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557860,"kb_used_data":363732,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93813980,"statfs":{"total":96636764160,"available":96065515520,"internally_reserved":0,"allocated":372461568,"data_stored":372202806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":198766249},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593749,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":345560,"kb_used_data":183368,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":94026280,"statfs":{"total":96636764160,"available":96282910720,"internally_reserved":0,"allocated":187768832,"data_stored":187440490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":166063790},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554724,"kb_used_data":364820,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93817116,"statfs":{"total":96636764160,"available":96068726784,"internally_reserved":0,"allocated":373575680,"data_stored":373316918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":194440874},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,2,5,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":374329344,"data_stored":374325865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188928000,"data_stored":188854941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":373551104,"data_stored":373547625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:22.089 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 145 20 115910 115890 3.12111 2.04297 0.0426756 0.0250252 2024-09-16T04:51:23.089 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 146 20 116445 116425 3.11404 2.08984 0.0174257 0.0250791 2024-09-16T04:51:24.089 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 147 20 116977 116957 3.107 2.07812 0.0922844 0.0251344 2024-09-16T04:51:24.872 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:25.089 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 148 20 117508 117488 3.10002 2.07422 0.023438 0.0251937 2024-09-16T04:51:25.925 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:25.926 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:25.959 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-16T04:51:24.317488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":475238642,"num_objects":116033,"num_object_clones":0,"num_object_copies":232066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":486,"num_objects_unfound":0,"num_objects_dirty":116033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81267,"num_write_kb":325034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2870,"num_bytes_recovered":11718675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116190,"ondisk_log_size":116190,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490664,"kb_used_data":936120,"kb_used_omap":12,"kb_used_meta":554483,"kb_avail":281624856,"statfs":{"total":289910292480,"available":288383852544,"internally_reserved":0,"allocated":958586880,"data_stored":957741014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":567790593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,5,6,8,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26873856,"num_objects":6561,"num_object_clones":0,"num_object_copies":13122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-409,"num_objects_unfound":0,"num_objects_dirty":6561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6561,"num_write_kb":26244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":409,"num_bytes_recovered":1675264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003003"},"pg_stats":[{"pgid":"2.f","version":"45'7165","reported_seq":7240,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:22.491195+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:22.491195+0000","last_peered":"2024-09-16T04:51:22.491195+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:22.491195+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7165,"log_dups_size":0,"ondisk_log_size":7165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7165,"num_write_kb":28660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3824},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7249","reported_seq":7361,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:21.986214+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:21.986214+0000","last_peered":"2024-09-16T04:51:21.986214+0000","last_clean":"2024-09-16T04:51:21.986214+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:21.986214+0000","last_undegraded":"2024-09-16T04:51:21.986214+0000","last_fullsized":"2024-09-16T04:51:21.986214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7249,"log_dups_size":0,"ondisk_log_size":7249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29667328,"num_objects":7243,"num_object_clones":0,"num_object_copies":14486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4218,"num_write_kb":16872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7187","reported_seq":7321,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:21.875961+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:21.875961+0000","last_peered":"2024-09-16T04:51:21.875961+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:21.875961+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7187,"log_dups_size":0,"ondisk_log_size":7187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29421568,"num_objects":7183,"num_object_clones":0,"num_object_copies":14366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3926,"num_write_kb":15704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":3864}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7285","reported_seq":7350,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:21.875950+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:21.875950+0000","last_peered":"2024-09-16T04:51:21.875950+0000","last_clean":"2024-09-16T04:51:21.875950+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:21.875950+0000","last_undegraded":"2024-09-16T04:51:21.875950+0000","last_fullsized":"2024-09-16T04:51:21.875950+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7285,"log_dups_size":0,"ondisk_log_size":7285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839360,"num_objects":7285,"num_object_clones":0,"num_object_copies":14570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7285,"num_write_kb":29140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7315","reported_seq":7529,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:22.565925+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:22.565925+0000","last_peered":"2024-09-16T04:51:22.565925+0000","last_clean":"2024-09-16T04:51:22.565925+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:22.565925+0000","last_undegraded":"2024-09-16T04:51:22.565925+0000","last_fullsized":"2024-09-16T04:51:22.565925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7315,"log_dups_size":0,"ondisk_log_size":7315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839360,"num_objects":7285,"num_object_clones":0,"num_object_copies":14570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4347,"num_write_kb":17388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7324","reported_seq":7448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:22.590482+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:22.590482+0000","last_peered":"2024-09-16T04:51:22.590482+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:22.590482+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7324,"log_dups_size":0,"ondisk_log_size":7324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29978624,"num_objects":7319,"num_object_clones":0,"num_object_copies":14638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4317,"num_write_kb":17268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3863},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7190","reported_seq":7334,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:22.502922+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:22.502922+0000","last_peered":"2024-09-16T04:51:22.502922+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:22.502922+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7190,"log_dups_size":0,"ondisk_log_size":7190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29425664,"num_objects":7184,"num_object_clones":0,"num_object_copies":14368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3845,"num_write_kb":15380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3785},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7257","reported_seq":7468,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:22.586285+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:22.586285+0000","last_peered":"2024-09-16T04:51:22.586285+0000","last_clean":"2024-09-16T04:51:22.586285+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:22.586285+0000","last_undegraded":"2024-09-16T04:51:22.586285+0000","last_fullsized":"2024-09-16T04:51:22.586285+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7257,"log_dups_size":0,"ondisk_log_size":7257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29691904,"num_objects":7249,"num_object_clones":0,"num_object_copies":14498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4327,"num_write_kb":17308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7240","reported_seq":7463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:21.985441+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:21.985441+0000","last_peered":"2024-09-16T04:51:21.985441+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:21.985441+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7240,"log_dups_size":0,"ondisk_log_size":7240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29519872,"num_objects":7207,"num_object_clones":0,"num_object_copies":14414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3991,"num_write_kb":15964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":3886}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7344","reported_seq":7467,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:22.569486+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:22.569486+0000","last_peered":"2024-09-16T04:51:22.569486+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:22.569486+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30048256,"num_objects":7336,"num_object_clones":0,"num_object_copies":14672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3999,"num_write_kb":15996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3871},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7164","reported_seq":7401,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:21.889139+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:21.889139+0000","last_peered":"2024-09-16T04:51:21.889139+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:21.889139+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7164,"log_dups_size":0,"ondisk_log_size":7164,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29218002,"num_objects":7135,"num_object_clones":0,"num_object_copies":14270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7136,"num_write_kb":28535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":3839}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7213","reported_seq":13085,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:51:21.981668+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:51:21.981668+0000","last_peered":"2024-09-16T04:51:21.981668+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:21.981668+0000","last_undegraded":"2024-09-16T04:51:21.981668+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7213,"log_dups_size":0,"ondisk_log_size":7213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29519872,"num_objects":7207,"num_object_clones":0,"num_object_copies":14414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":486,"num_objects_unfound":0,"num_objects_dirty":7207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3936,"num_write_kb":15744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2860,"num_bytes_recovered":11706368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":486},{"shards":"2","objects":6721}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7143","reported_seq":7236,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:21.909351+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:21.909351+0000","last_peered":"2024-09-16T04:51:21.909351+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:21.909351+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7143,"log_dups_size":0,"ondisk_log_size":7143,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29250153,"num_objects":7142,"num_object_clones":0,"num_object_copies":14284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7142,"num_write_kb":28565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":3735}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7412","reported_seq":7509,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:21.956281+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:21.956281+0000","last_peered":"2024-09-16T04:51:21.956281+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:21.956281+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7412,"log_dups_size":0,"ondisk_log_size":7412,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30328635,"num_objects":7407,"num_object_clones":0,"num_object_copies":14814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7409,"num_write_kb":29620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":3893}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7300","reported_seq":7427,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:22.572150+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:22.572150+0000","last_peered":"2024-09-16T04:51:22.572150+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:22.572150+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7300,"log_dups_size":0,"ondisk_log_size":7300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4054,"num_write_kb":16216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3971},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7399","reported_seq":7533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:22.602862+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:22.602862+0000","last_peered":"2024-09-16T04:51:22.602862+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:22.602862+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30257769,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4168,"num_write_kb":16672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3864},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":475238623,"num_objects":116031,"num_object_clones":0,"num_object_copies":232062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":486,"num_objects_unfound":0,"num_objects_dirty":116031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81265,"num_write_kb":325032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2869,"num_bytes_recovered":11718656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":958910464,"data_stored":958830447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116187,"ondisk_log_size":116187,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573912,"kb_used_data":371464,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93797928,"statfs":{"total":96636764160,"available":96049078272,"internally_reserved":0,"allocated":380379136,"data_stored":380120374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":207285929},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593750,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347388,"kb_used_data":193516,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94024452,"statfs":{"total":96636764160,"available":96281038848,"internally_reserved":0,"allocated":198160384,"data_stored":197832042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569364,"kb_used_data":371140,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93802476,"statfs":{"total":96636764160,"available":96053735424,"internally_reserved":0,"allocated":380047360,"data_stored":379788598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":202960554},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,5,0,3,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":380911616,"data_stored":380908137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":197910528,"data_stored":197837469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380088320,"data_stored":380084841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:25.961 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:26.089 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 149 20 118014 117994 3.09248 1.97656 0.0770536 0.0252553 2024-09-16T04:51:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:26.857 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:26.906 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-16T04:51:26.317957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":475238642,"num_objects":116033,"num_object_clones":0,"num_object_copies":232066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":486,"num_objects_unfound":0,"num_objects_dirty":116033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81267,"num_write_kb":325034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2870,"num_bytes_recovered":11718675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116190,"ondisk_log_size":116190,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1503648,"kb_used_data":944944,"kb_used_omap":12,"kb_used_meta":558643,"kb_avail":281611872,"statfs":{"total":289910292480,"available":288370556928,"internally_reserved":0,"allocated":967622656,"data_stored":966776790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":572050433},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,5,4,11,5,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21270528,"num_objects":5193,"num_object_clones":0,"num_object_copies":10386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-409,"num_objects_unfound":0,"num_objects_dirty":5193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5193,"num_write_kb":20772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":409,"num_bytes_recovered":1675264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002992"},"pg_stats":[{"pgid":"2.f","version":"45'7165","reported_seq":7240,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:22.491195+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:22.491195+0000","last_peered":"2024-09-16T04:51:22.491195+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:22.491195+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7165,"log_dups_size":0,"ondisk_log_size":7165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7165,"num_write_kb":28660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3824},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7249","reported_seq":7361,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:21.986214+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:21.986214+0000","last_peered":"2024-09-16T04:51:21.986214+0000","last_clean":"2024-09-16T04:51:21.986214+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:21.986214+0000","last_undegraded":"2024-09-16T04:51:21.986214+0000","last_fullsized":"2024-09-16T04:51:21.986214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7249,"log_dups_size":0,"ondisk_log_size":7249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29667328,"num_objects":7243,"num_object_clones":0,"num_object_copies":14486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4218,"num_write_kb":16872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7187","reported_seq":7321,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:21.875961+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:21.875961+0000","last_peered":"2024-09-16T04:51:21.875961+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:21.875961+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7187,"log_dups_size":0,"ondisk_log_size":7187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29421568,"num_objects":7183,"num_object_clones":0,"num_object_copies":14366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3926,"num_write_kb":15704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":3864}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7285","reported_seq":7350,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:21.875950+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:21.875950+0000","last_peered":"2024-09-16T04:51:21.875950+0000","last_clean":"2024-09-16T04:51:21.875950+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:21.875950+0000","last_undegraded":"2024-09-16T04:51:21.875950+0000","last_fullsized":"2024-09-16T04:51:21.875950+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7285,"log_dups_size":0,"ondisk_log_size":7285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839360,"num_objects":7285,"num_object_clones":0,"num_object_copies":14570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7285,"num_write_kb":29140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7315","reported_seq":7529,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:22.565925+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:22.565925+0000","last_peered":"2024-09-16T04:51:22.565925+0000","last_clean":"2024-09-16T04:51:22.565925+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:22.565925+0000","last_undegraded":"2024-09-16T04:51:22.565925+0000","last_fullsized":"2024-09-16T04:51:22.565925+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7315,"log_dups_size":0,"ondisk_log_size":7315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839360,"num_objects":7285,"num_object_clones":0,"num_object_copies":14570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4347,"num_write_kb":17388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7324","reported_seq":7448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:22.590482+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:22.590482+0000","last_peered":"2024-09-16T04:51:22.590482+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:22.590482+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7324,"log_dups_size":0,"ondisk_log_size":7324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29978624,"num_objects":7319,"num_object_clones":0,"num_object_copies":14638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4317,"num_write_kb":17268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3863},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7190","reported_seq":7334,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:22.502922+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:22.502922+0000","last_peered":"2024-09-16T04:51:22.502922+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:22.502922+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7190,"log_dups_size":0,"ondisk_log_size":7190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29425664,"num_objects":7184,"num_object_clones":0,"num_object_copies":14368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3845,"num_write_kb":15380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3785},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7257","reported_seq":7468,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:22.586285+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:22.586285+0000","last_peered":"2024-09-16T04:51:22.586285+0000","last_clean":"2024-09-16T04:51:22.586285+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:22.586285+0000","last_undegraded":"2024-09-16T04:51:22.586285+0000","last_fullsized":"2024-09-16T04:51:22.586285+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7257,"log_dups_size":0,"ondisk_log_size":7257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29691904,"num_objects":7249,"num_object_clones":0,"num_object_copies":14498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4327,"num_write_kb":17308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7240","reported_seq":7463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:21.985441+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:21.985441+0000","last_peered":"2024-09-16T04:51:21.985441+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:21.985441+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7240,"log_dups_size":0,"ondisk_log_size":7240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29519872,"num_objects":7207,"num_object_clones":0,"num_object_copies":14414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3991,"num_write_kb":15964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":3886}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7344","reported_seq":7467,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:22.569486+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:22.569486+0000","last_peered":"2024-09-16T04:51:22.569486+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:22.569486+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7344,"log_dups_size":0,"ondisk_log_size":7344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30048256,"num_objects":7336,"num_object_clones":0,"num_object_copies":14672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3999,"num_write_kb":15996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3871},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7164","reported_seq":7401,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:21.889139+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:21.889139+0000","last_peered":"2024-09-16T04:51:21.889139+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:21.889139+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7164,"log_dups_size":0,"ondisk_log_size":7164,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29218002,"num_objects":7135,"num_object_clones":0,"num_object_copies":14270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7136,"num_write_kb":28535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":3839}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7213","reported_seq":13085,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:51:21.981668+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:51:21.981668+0000","last_peered":"2024-09-16T04:51:21.981668+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:21.981668+0000","last_undegraded":"2024-09-16T04:51:21.981668+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7213,"log_dups_size":0,"ondisk_log_size":7213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29519872,"num_objects":7207,"num_object_clones":0,"num_object_copies":14414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":486,"num_objects_unfound":0,"num_objects_dirty":7207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3936,"num_write_kb":15744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2860,"num_bytes_recovered":11706368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":486},{"shards":"2","objects":6721}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7143","reported_seq":7236,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:21.909351+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:21.909351+0000","last_peered":"2024-09-16T04:51:21.909351+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:21.909351+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7143,"log_dups_size":0,"ondisk_log_size":7143,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29250153,"num_objects":7142,"num_object_clones":0,"num_object_copies":14284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7142,"num_write_kb":28565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":3735}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7412","reported_seq":7509,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:21.956281+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:21.956281+0000","last_peered":"2024-09-16T04:51:21.956281+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:21.956281+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7412,"log_dups_size":0,"ondisk_log_size":7412,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30328635,"num_objects":7407,"num_object_clones":0,"num_object_copies":14814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7409,"num_write_kb":29620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":3893}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7300","reported_seq":7427,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:22.572150+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:22.572150+0000","last_peered":"2024-09-16T04:51:22.572150+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:22.572150+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7300,"log_dups_size":0,"ondisk_log_size":7300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4054,"num_write_kb":16216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3971},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7399","reported_seq":7533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:22.602862+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:22.602862+0000","last_peered":"2024-09-16T04:51:22.602862+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:22.602862+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30257769,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4168,"num_write_kb":16672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3864},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":475238623,"num_objects":116031,"num_object_clones":0,"num_object_copies":232062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":486,"num_objects_unfound":0,"num_objects_dirty":116031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81265,"num_write_kb":325032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2869,"num_bytes_recovered":11718656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":967995392,"data_stored":967915375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116187,"ondisk_log_size":116187,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573912,"kb_used_data":371464,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93797928,"statfs":{"total":96636764160,"available":96049078272,"internally_reserved":0,"allocated":380379136,"data_stored":380120374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":207285929},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593751,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":360372,"kb_used_data":202340,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94011468,"statfs":{"total":96636764160,"available":96267743232,"internally_reserved":0,"allocated":207196160,"data_stored":206867818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":161803950},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,3,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569364,"kb_used_data":371140,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93802476,"statfs":{"total":96636764160,"available":96053735424,"internally_reserved":0,"allocated":380047360,"data_stored":379788598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":202960554},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,5,0,3,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":380911616,"data_stored":380908137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206995456,"data_stored":206922397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380088320,"data_stored":380084841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:26.907 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:51:26.907 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'7165", 'reported_seq': 7240, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:22.491195+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:51:22.491195+0000', 'last_peered': '2024-09-16T04:51:22.491195+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:51:22.491195+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 7165, 'log_dups_size': 0, 'ondisk_log_size': 7165, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29347840, 'num_objects': 7165, 'num_object_clones': 0, 'num_object_copies': 14330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7165, 'num_write_kb': 28660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3824}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:26.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:51:26.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'7187", 'reported_seq': 7321, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:21.875961+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:51:21.875961+0000', 'last_peered': '2024-09-16T04:51:21.875961+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:51:21.875961+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 7187, 'log_dups_size': 0, 'ondisk_log_size': 7187, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29421568, 'num_objects': 7183, 'num_object_clones': 0, 'num_object_copies': 14366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7183, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3926, 'num_write_kb': 15704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 3864}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:26.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:51:26.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'7324", 'reported_seq': 7448, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:22.590482+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:51:22.590482+0000', 'last_peered': '2024-09-16T04:51:22.590482+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:51:22.590482+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 7324, 'log_dups_size': 0, 'ondisk_log_size': 7324, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29978624, 'num_objects': 7319, 'num_object_clones': 0, 'num_object_copies': 14638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7319, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4317, 'num_write_kb': 17268, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3863}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:26.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:51:26.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'7190", 'reported_seq': 7334, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:22.502922+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:51:22.502922+0000', 'last_peered': '2024-09-16T04:51:22.502922+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:51:22.502922+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 7190, 'log_dups_size': 0, 'ondisk_log_size': 7190, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29425664, 'num_objects': 7184, 'num_object_clones': 0, 'num_object_copies': 14368, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7184, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3845, 'num_write_kb': 15380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3785}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:26.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:51:26.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'7240", 'reported_seq': 7463, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:21.985441+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:51:21.985441+0000', 'last_peered': '2024-09-16T04:51:21.985441+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:51:21.985441+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 7240, 'log_dups_size': 0, 'ondisk_log_size': 7240, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29519872, 'num_objects': 7207, 'num_object_clones': 0, 'num_object_copies': 14414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7207, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3991, 'num_write_kb': 15964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 3886}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:26.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:51:26.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'7344", 'reported_seq': 7467, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:22.569486+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:51:22.569486+0000', 'last_peered': '2024-09-16T04:51:22.569486+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:51:22.569486+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 7344, 'log_dups_size': 0, 'ondisk_log_size': 7344, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30048256, 'num_objects': 7336, 'num_object_clones': 0, 'num_object_copies': 14672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7336, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3999, 'num_write_kb': 15996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3871}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:26.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:51:26.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'7164", 'reported_seq': 7401, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:21.889139+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:51:21.889139+0000', 'last_peered': '2024-09-16T04:51:21.889139+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:51:21.889139+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 7164, 'log_dups_size': 0, 'ondisk_log_size': 7164, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29218002, 'num_objects': 7135, 'num_object_clones': 0, 'num_object_copies': 14270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7135, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7136, 'num_write_kb': 28535, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 3839}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:26.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:51:26.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'7213", 'reported_seq': 13085, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-16T04:51:21.981668+0000', 'last_change': '2024-09-16T04:49:38.054133+0000', 'last_active': '2024-09-16T04:51:21.981668+0000', 'last_peered': '2024-09-16T04:51:21.981668+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:51:21.981668+0000', 'last_undegraded': '2024-09-16T04:51:21.981668+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 7213, 'log_dups_size': 0, 'ondisk_log_size': 7213, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29519872, 'num_objects': 7207, 'num_object_clones': 0, 'num_object_copies': 14414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 486, 'num_objects_unfound': 0, 'num_objects_dirty': 7207, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3936, 'num_write_kb': 15744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2860, 'num_bytes_recovered': 11706368, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 486}, {'shards': '2', 'objects': 6721}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:26.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:51:26.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'7143", 'reported_seq': 7236, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:21.909351+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:51:21.909351+0000', 'last_peered': '2024-09-16T04:51:21.909351+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:51:21.909351+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 7143, 'log_dups_size': 0, 'ondisk_log_size': 7143, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29250153, 'num_objects': 7142, 'num_object_clones': 0, 'num_object_copies': 14284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7142, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7142, 'num_write_kb': 28565, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 3735}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:26.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:51:26.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'7412", 'reported_seq': 7509, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:21.956281+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:51:21.956281+0000', 'last_peered': '2024-09-16T04:51:21.956281+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:51:21.956281+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 7412, 'log_dups_size': 0, 'ondisk_log_size': 7412, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30328635, 'num_objects': 7407, 'num_object_clones': 0, 'num_object_copies': 14814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7409, 'num_write_kb': 29620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 3893}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:26.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:51:26.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'7300", 'reported_seq': 7427, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:22.572150+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:51:22.572150+0000', 'last_peered': '2024-09-16T04:51:22.572150+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:51:22.572150+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 7300, 'log_dups_size': 0, 'ondisk_log_size': 7300, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29884416, 'num_objects': 7296, 'num_object_clones': 0, 'num_object_copies': 14592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4054, 'num_write_kb': 16216, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3971}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:26.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:51:26.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'7399", 'reported_seq': 7533, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:22.602862+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:51:22.602862+0000', 'last_peered': '2024-09-16T04:51:22.602862+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:51:22.602862+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 7399, 'log_dups_size': 0, 'ondisk_log_size': 7399, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30257769, 'num_objects': 7388, 'num_object_clones': 0, 'num_object_copies': 14776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4168, 'num_write_kb': 16672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3864}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:26.910 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:51:27.089 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 150 20 118494 118474 3.08436 1.875 0.0849178 0.0253213 2024-09-16T04:51:27.613 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:27.613 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/232066 objects degraded (16.111%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":235,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":116033,"data_bytes":475238642,"bytes_used":1539735552,"bytes_avail":288370556928,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":232066,"degraded_ratio":0.16110503046547101,"misplaced_objects":486,"misplaced_total":232066,"misplaced_ratio":0.0020942318133634398,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":139570,"recovering_keys_per_sec":0,"num_objects_recovered":409,"num_bytes_recovered":1675264,"num_keys_recovered":0,"write_bytes_sec":1772102,"read_op_per_sec":0,"write_op_per_sec":432},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:51:27.660 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:51:27.661 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:28.089 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 151 20 118929 118909 3.07519 1.69922 0.0181855 0.0253945 2024-09-16T04:51:28.387 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:28.388 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:28.444 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-16T04:51:28.318423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":485687538,"num_objects":118584,"num_object_clones":0,"num_object_copies":237168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":242,"num_objects_unfound":0,"num_objects_dirty":118584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83818,"num_write_kb":335238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3114,"num_bytes_recovered":12718099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118741,"ondisk_log_size":118741,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1508368,"kb_used_data":957984,"kb_used_omap":12,"kb_used_meta":550323,"kb_avail":281607152,"statfs":{"total":289910292480,"available":288365723648,"internally_reserved":0,"allocated":980975616,"data_stored":980129750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":563530753},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,5,9,7,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31719424,"num_objects":7744,"num_object_clones":0,"num_object_copies":15488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-653,"num_objects_unfound":0,"num_objects_dirty":7744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7744,"num_write_kb":30976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":653,"num_bytes_recovered":2674688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003002"},"pg_stats":[{"pgid":"2.f","version":"45'7296","reported_seq":7371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.493931+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:27.493931+0000","last_peered":"2024-09-16T04:51:27.493931+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:27.493931+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7296,"log_dups_size":0,"ondisk_log_size":7296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7296,"num_write_kb":29184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3955},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7385","reported_seq":7497,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:26.939876+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:26.939876+0000","last_peered":"2024-09-16T04:51:26.939876+0000","last_clean":"2024-09-16T04:51:26.939876+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:26.939876+0000","last_undegraded":"2024-09-16T04:51:26.939876+0000","last_fullsized":"2024-09-16T04:51:26.939876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7385,"log_dups_size":0,"ondisk_log_size":7385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30224384,"num_objects":7379,"num_object_clones":0,"num_object_copies":14758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4354,"num_write_kb":17416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7336","reported_seq":7470,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:26.967782+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:26.967782+0000","last_peered":"2024-09-16T04:51:26.967782+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:26.967782+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7336,"log_dups_size":0,"ondisk_log_size":7336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30031872,"num_objects":7332,"num_object_clones":0,"num_object_copies":14664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4075,"num_write_kb":16300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4013}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7456","reported_seq":7522,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:27.015783+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:27.015783+0000","last_peered":"2024-09-16T04:51:27.015783+0000","last_clean":"2024-09-16T04:51:27.015783+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:27.015783+0000","last_undegraded":"2024-09-16T04:51:27.015783+0000","last_fullsized":"2024-09-16T04:51:27.015783+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7456,"log_dups_size":0,"ondisk_log_size":7456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30539776,"num_objects":7456,"num_object_clones":0,"num_object_copies":14912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7456,"num_write_kb":29824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7488","reported_seq":7701,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:27.658692+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:27.658692+0000","last_peered":"2024-09-16T04:51:27.658692+0000","last_clean":"2024-09-16T04:51:27.658692+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:27.658692+0000","last_undegraded":"2024-09-16T04:51:27.658692+0000","last_fullsized":"2024-09-16T04:51:27.658692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7488,"log_dups_size":0,"ondisk_log_size":7488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30547968,"num_objects":7458,"num_object_clones":0,"num_object_copies":14916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4520,"num_write_kb":18080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7487","reported_seq":7616,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.607462+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:27.607462+0000","last_peered":"2024-09-16T04:51:27.607462+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:27.607462+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7487,"log_dups_size":0,"ondisk_log_size":7487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30646272,"num_objects":7482,"num_object_clones":0,"num_object_copies":14964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4480,"num_write_kb":17920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4026},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7356","reported_seq":7500,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.581206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:27.581206+0000","last_peered":"2024-09-16T04:51:27.581206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:27.581206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7356,"log_dups_size":0,"ondisk_log_size":7356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30105600,"num_objects":7350,"num_object_clones":0,"num_object_copies":14700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4011,"num_write_kb":16044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3951},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7399","reported_seq":7610,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:27.542687+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:27.542687+0000","last_peered":"2024-09-16T04:51:27.542687+0000","last_clean":"2024-09-16T04:51:27.542687+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:27.542687+0000","last_undegraded":"2024-09-16T04:51:27.542687+0000","last_fullsized":"2024-09-16T04:51:27.542687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30273536,"num_objects":7391,"num_object_clones":0,"num_object_copies":14782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4469,"num_write_kb":17876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7409","reported_seq":7632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:26.990475+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:26.990475+0000","last_peered":"2024-09-16T04:51:26.990475+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:26.990475+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7409,"log_dups_size":0,"ondisk_log_size":7409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30212096,"num_objects":7376,"num_object_clones":0,"num_object_copies":14752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4160,"num_write_kb":16640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":4055}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7499","reported_seq":7624,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.597337+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:27.597337+0000","last_peered":"2024-09-16T04:51:27.597337+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:27.597337+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7499,"log_dups_size":0,"ondisk_log_size":7499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30683136,"num_objects":7491,"num_object_clones":0,"num_object_copies":14982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4154,"num_write_kb":16616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4026},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7338","reported_seq":7575,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:26.963811+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:26.963811+0000","last_peered":"2024-09-16T04:51:26.963811+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:26.963811+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7338,"log_dups_size":0,"ondisk_log_size":7338,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29930706,"num_objects":7309,"num_object_clones":0,"num_object_copies":14618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7310,"num_write_kb":29231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4013}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7383","reported_seq":13745,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T04:51:27.000819+0000","last_change":"2024-09-16T04:49:38.054133+0000","last_active":"2024-09-16T04:51:27.000819+0000","last_peered":"2024-09-16T04:51:27.000819+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:27.000819+0000","last_undegraded":"2024-09-16T04:51:27.000819+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7383,"log_dups_size":0,"ondisk_log_size":7383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30216192,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":242,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4106,"num_write_kb":16424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3104,"num_bytes_recovered":12705792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":242},{"shards":"2","objects":7135}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7293","reported_seq":7386,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:26.960088+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:26.960088+0000","last_peered":"2024-09-16T04:51:26.960088+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:26.960088+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7293,"log_dups_size":0,"ondisk_log_size":7293,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29864553,"num_objects":7292,"num_object_clones":0,"num_object_copies":14584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7292,"num_write_kb":29165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":3885}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7578","reported_seq":7675,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.003734+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:27.003734+0000","last_peered":"2024-09-16T04:51:27.003734+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:27.003734+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7578,"log_dups_size":0,"ondisk_log_size":7578,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31008571,"num_objects":7573,"num_object_clones":0,"num_object_copies":15146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7575,"num_write_kb":30284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4059}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7463","reported_seq":7590,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.613507+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:27.613507+0000","last_peered":"2024-09-16T04:51:27.613507+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:27.613507+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7463,"log_dups_size":0,"ondisk_log_size":7463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30552064,"num_objects":7459,"num_object_clones":0,"num_object_copies":14918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4217,"num_write_kb":16868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4134},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7572","reported_seq":7707,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.614460+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:27.614460+0000","last_peered":"2024-09-16T04:51:27.614460+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:27.614460+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":7572,"log_dups_size":0,"ondisk_log_size":7572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30966377,"num_objects":7561,"num_object_clones":0,"num_object_copies":15122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4341,"num_write_kb":17364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4037},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":485687519,"num_objects":118582,"num_object_clones":0,"num_object_copies":237164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":242,"num_objects_unfound":0,"num_objects_dirty":118582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83816,"num_write_kb":335236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3113,"num_bytes_recovered":12718080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":981106688,"data_stored":981026671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118738,"ondisk_log_size":118738,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567900,"kb_used_data":377932,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93803940,"statfs":{"total":96636764160,"available":96055234560,"internally_reserved":0,"allocated":387002368,"data_stored":386743606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":194506409},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593751,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":360372,"kb_used_data":202340,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94011468,"statfs":{"total":96636764160,"available":96267743232,"internally_reserved":0,"allocated":207196160,"data_stored":206867818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":161803950},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,3,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580096,"kb_used_data":377712,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93791744,"statfs":{"total":96636764160,"available":96042745856,"internally_reserved":0,"allocated":386777088,"data_stored":386518326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":207220394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,1,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":387350528,"data_stored":387347049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206995456,"data_stored":206922397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386760704,"data_stored":386757225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:29.090 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 152 20 119440 119420 3.06809 1.99609 0.0103209 0.0254562 2024-09-16T04:51:30.090 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 153 20 119967 119947 3.06149 2.05859 0.0106253 0.0255091 2024-09-16T04:51:31.091 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 154 20 120508 120488 3.05531 2.11328 0.00967951 0.0255616 2024-09-16T04:51:31.447 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:32.091 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 155 20 120990 120970 3.04774 1.88281 0.0894281 0.0256224 2024-09-16T04:51:32.317 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:32.319 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:32.397 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-16T04:51:32.139991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":491032818,"num_objects":119889,"num_object_clones":0,"num_object_copies":239778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85123,"num_write_kb":340458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3356,"num_bytes_recovered":13709331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120046,"ondisk_log_size":120046,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1553572,"kb_used_data":972660,"kb_used_omap":12,"kb_used_meta":580851,"kb_avail":281561948,"statfs":{"total":289910292480,"available":288319434752,"internally_reserved":0,"allocated":996003840,"data_stored":995157974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":594791425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,5,3,9,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-701,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6437,"num_write_kb":25748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":701,"num_bytes_recovered":2871296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.823462"},"pg_stats":[{"pgid":"2.f","version":"45'7296","reported_seq":7371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.493931+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:27.493931+0000","last_peered":"2024-09-16T04:51:27.493931+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:27.493931+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7296,"log_dups_size":0,"ondisk_log_size":7296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7296,"num_write_kb":29184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3955},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7537","reported_seq":7649,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:31.956472+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:31.956472+0000","last_peered":"2024-09-16T04:51:31.956472+0000","last_clean":"2024-09-16T04:51:31.956472+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:31.956472+0000","last_undegraded":"2024-09-16T04:51:31.956472+0000","last_fullsized":"2024-09-16T04:51:31.956472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7537,"log_dups_size":0,"ondisk_log_size":7537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30846976,"num_objects":7531,"num_object_clones":0,"num_object_copies":15062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4506,"num_write_kb":18024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7498","reported_seq":7632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:31.910878+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:31.910878+0000","last_peered":"2024-09-16T04:51:31.910878+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:31.910878+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7498,"log_dups_size":0,"ondisk_log_size":7498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30695424,"num_objects":7494,"num_object_clones":0,"num_object_copies":14988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4237,"num_write_kb":16948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4175}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7629","reported_seq":7695,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:31.997617+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:31.997617+0000","last_peered":"2024-09-16T04:51:31.997617+0000","last_clean":"2024-09-16T04:51:31.997617+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:31.997617+0000","last_undegraded":"2024-09-16T04:51:31.997617+0000","last_fullsized":"2024-09-16T04:51:31.997617+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7629,"log_dups_size":0,"ondisk_log_size":7629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31248384,"num_objects":7629,"num_object_clones":0,"num_object_copies":15258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7629,"num_write_kb":30516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7488","reported_seq":7701,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:27.658692+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:27.658692+0000","last_peered":"2024-09-16T04:51:27.658692+0000","last_clean":"2024-09-16T04:51:27.658692+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:27.658692+0000","last_undegraded":"2024-09-16T04:51:27.658692+0000","last_fullsized":"2024-09-16T04:51:27.658692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7488,"log_dups_size":0,"ondisk_log_size":7488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30547968,"num_objects":7458,"num_object_clones":0,"num_object_copies":14916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4520,"num_write_kb":18080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7487","reported_seq":7616,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.607462+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:27.607462+0000","last_peered":"2024-09-16T04:51:27.607462+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:27.607462+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7487,"log_dups_size":0,"ondisk_log_size":7487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30646272,"num_objects":7482,"num_object_clones":0,"num_object_copies":14964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4480,"num_write_kb":17920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4026},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7356","reported_seq":7500,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.581206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:27.581206+0000","last_peered":"2024-09-16T04:51:27.581206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:27.581206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7356,"log_dups_size":0,"ondisk_log_size":7356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30105600,"num_objects":7350,"num_object_clones":0,"num_object_copies":14700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4011,"num_write_kb":16044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3951},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7399","reported_seq":7610,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:27.542687+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:27.542687+0000","last_peered":"2024-09-16T04:51:27.542687+0000","last_clean":"2024-09-16T04:51:27.542687+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:27.542687+0000","last_undegraded":"2024-09-16T04:51:27.542687+0000","last_fullsized":"2024-09-16T04:51:27.542687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30273536,"num_objects":7391,"num_object_clones":0,"num_object_copies":14782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4469,"num_write_kb":17876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7591","reported_seq":7814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:31.924429+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:31.924429+0000","last_peered":"2024-09-16T04:51:31.924429+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:31.924429+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7591,"log_dups_size":0,"ondisk_log_size":7591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30957568,"num_objects":7558,"num_object_clones":0,"num_object_copies":15116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4342,"num_write_kb":17368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7499","reported_seq":7624,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.597337+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:27.597337+0000","last_peered":"2024-09-16T04:51:27.597337+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:27.597337+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7499,"log_dups_size":0,"ondisk_log_size":7499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30683136,"num_objects":7491,"num_object_clones":0,"num_object_copies":14982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4154,"num_write_kb":16616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4026},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7494","reported_seq":7732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:31.987286+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:31.987286+0000","last_peered":"2024-09-16T04:51:31.987286+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:31.987286+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7494,"log_dups_size":0,"ondisk_log_size":7494,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30569682,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7466,"num_write_kb":29855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4169}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7545","reported_seq":14392,"reported_epoch":45,"state":"active+undersized+remapped","last_fresh":"2024-09-16T04:51:31.963292+0000","last_change":"2024-09-16T04:51:31.738291+0000","last_active":"2024-09-16T04:51:31.963292+0000","last_peered":"2024-09-16T04:51:31.963292+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:31.963292+0000","last_undegraded":"2024-09-16T04:51:31.963292+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7545,"log_dups_size":0,"ondisk_log_size":7545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30879744,"num_objects":7539,"num_object_clones":0,"num_object_copies":15078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4268,"num_write_kb":17072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":7539}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7466","reported_seq":7554,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:32.034646+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:32.034646+0000","last_peered":"2024-09-16T04:51:32.034646+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:32.034646+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7466,"log_dups_size":0,"ondisk_log_size":7466,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30573161,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7465,"num_write_kb":29857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7723","reported_seq":7821,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:31.943239+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:31.943239+0000","last_peered":"2024-09-16T04:51:31.943239+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:31.943239+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7723,"log_dups_size":0,"ondisk_log_size":7723,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31602491,"num_objects":7718,"num_object_clones":0,"num_object_copies":15436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7720,"num_write_kb":30864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4204}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7463","reported_seq":7590,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.613507+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:27.613507+0000","last_peered":"2024-09-16T04:51:27.613507+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:27.613507+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7463,"log_dups_size":0,"ondisk_log_size":7463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30552064,"num_objects":7459,"num_object_clones":0,"num_object_copies":14918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4217,"num_write_kb":16868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4134},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7572","reported_seq":7707,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.614460+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:27.614460+0000","last_peered":"2024-09-16T04:51:27.614460+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:27.614460+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":7572,"log_dups_size":0,"ondisk_log_size":7572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30966377,"num_objects":7561,"num_object_clones":0,"num_object_copies":15122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4341,"num_write_kb":17364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4037},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":491032799,"num_objects":119887,"num_object_clones":0,"num_object_copies":239774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85121,"num_write_kb":340456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3355,"num_bytes_recovered":13709312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":996323328,"data_stored":996243311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120043,"ondisk_log_size":120043,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596504,"kb_used_data":384328,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93775336,"statfs":{"total":96636764160,"available":96025944064,"internally_reserved":0,"allocated":393551872,"data_stored":393293110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":217247401},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,0,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593752,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376972,"kb_used_data":210620,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93994868,"statfs":{"total":96636764160,"available":96250744832,"internally_reserved":0,"allocated":215674880,"data_stored":215346538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170323630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,6,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580096,"kb_used_data":377712,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93791744,"statfs":{"total":96636764160,"available":96042745856,"internally_reserved":0,"allocated":386777088,"data_stored":386518326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":207220394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,1,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":387350528,"data_stored":387347049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":215678976,"data_stored":215605917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":393293824,"data_stored":393290345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:32.399 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:33.013 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:33.042 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-16T04:51:32.319359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":491032818,"num_objects":119889,"num_object_clones":0,"num_object_copies":239778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85123,"num_write_kb":340458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3356,"num_bytes_recovered":13709331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120046,"ondisk_log_size":120046,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1553572,"kb_used_data":972660,"kb_used_omap":12,"kb_used_meta":580851,"kb_avail":281561948,"statfs":{"total":289910292480,"available":288319434752,"internally_reserved":0,"allocated":996003840,"data_stored":995157974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":594791425},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,5,3,9,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21155840,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-486,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5165,"num_write_kb":20660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":486,"num_bytes_recovered":1990656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002346"},"pg_stats":[{"pgid":"2.f","version":"45'7296","reported_seq":7371,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.493931+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:27.493931+0000","last_peered":"2024-09-16T04:51:27.493931+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:27.493931+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7296,"log_dups_size":0,"ondisk_log_size":7296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7296,"num_write_kb":29184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3955},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7537","reported_seq":7649,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:31.956472+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:31.956472+0000","last_peered":"2024-09-16T04:51:31.956472+0000","last_clean":"2024-09-16T04:51:31.956472+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:31.956472+0000","last_undegraded":"2024-09-16T04:51:31.956472+0000","last_fullsized":"2024-09-16T04:51:31.956472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7537,"log_dups_size":0,"ondisk_log_size":7537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30846976,"num_objects":7531,"num_object_clones":0,"num_object_copies":15062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4506,"num_write_kb":18024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7498","reported_seq":7632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:31.910878+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:31.910878+0000","last_peered":"2024-09-16T04:51:31.910878+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:31.910878+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7498,"log_dups_size":0,"ondisk_log_size":7498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30695424,"num_objects":7494,"num_object_clones":0,"num_object_copies":14988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4237,"num_write_kb":16948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4175}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7629","reported_seq":7695,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:31.997617+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:31.997617+0000","last_peered":"2024-09-16T04:51:31.997617+0000","last_clean":"2024-09-16T04:51:31.997617+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:31.997617+0000","last_undegraded":"2024-09-16T04:51:31.997617+0000","last_fullsized":"2024-09-16T04:51:31.997617+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7629,"log_dups_size":0,"ondisk_log_size":7629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31248384,"num_objects":7629,"num_object_clones":0,"num_object_copies":15258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7629,"num_write_kb":30516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'7488","reported_seq":7701,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:27.658692+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:27.658692+0000","last_peered":"2024-09-16T04:51:27.658692+0000","last_clean":"2024-09-16T04:51:27.658692+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:27.658692+0000","last_undegraded":"2024-09-16T04:51:27.658692+0000","last_fullsized":"2024-09-16T04:51:27.658692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7488,"log_dups_size":0,"ondisk_log_size":7488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30547968,"num_objects":7458,"num_object_clones":0,"num_object_copies":14916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4520,"num_write_kb":18080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'7487","reported_seq":7616,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.607462+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:27.607462+0000","last_peered":"2024-09-16T04:51:27.607462+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:27.607462+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7487,"log_dups_size":0,"ondisk_log_size":7487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30646272,"num_objects":7482,"num_object_clones":0,"num_object_copies":14964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4480,"num_write_kb":17920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4026},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'7356","reported_seq":7500,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.581206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:27.581206+0000","last_peered":"2024-09-16T04:51:27.581206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:27.581206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7356,"log_dups_size":0,"ondisk_log_size":7356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30105600,"num_objects":7350,"num_object_clones":0,"num_object_copies":14700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4011,"num_write_kb":16044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3951},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'7399","reported_seq":7610,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:27.542687+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:27.542687+0000","last_peered":"2024-09-16T04:51:27.542687+0000","last_clean":"2024-09-16T04:51:27.542687+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:27.542687+0000","last_undegraded":"2024-09-16T04:51:27.542687+0000","last_fullsized":"2024-09-16T04:51:27.542687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30273536,"num_objects":7391,"num_object_clones":0,"num_object_copies":14782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4469,"num_write_kb":17876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7591","reported_seq":7814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:31.924429+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:31.924429+0000","last_peered":"2024-09-16T04:51:31.924429+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:31.924429+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7591,"log_dups_size":0,"ondisk_log_size":7591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30957568,"num_objects":7558,"num_object_clones":0,"num_object_copies":15116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4342,"num_write_kb":17368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'7499","reported_seq":7624,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.597337+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:27.597337+0000","last_peered":"2024-09-16T04:51:27.597337+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:27.597337+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7499,"log_dups_size":0,"ondisk_log_size":7499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30683136,"num_objects":7491,"num_object_clones":0,"num_object_copies":14982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4154,"num_write_kb":16616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4026},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.243752+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:49:38.243752+0000","last_peered":"2024-09-16T04:49:38.243752+0000","last_clean":"2024-09-16T04:49:38.243752+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:49:38.243752+0000","last_undegraded":"2024-09-16T04:49:38.243752+0000","last_fullsized":"2024-09-16T04:49:38.243752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7494","reported_seq":7732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:31.987286+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:31.987286+0000","last_peered":"2024-09-16T04:51:31.987286+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:31.987286+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7494,"log_dups_size":0,"ondisk_log_size":7494,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30569682,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7466,"num_write_kb":29855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4169}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7545","reported_seq":14392,"reported_epoch":45,"state":"active+undersized+remapped","last_fresh":"2024-09-16T04:51:31.963292+0000","last_change":"2024-09-16T04:51:31.738291+0000","last_active":"2024-09-16T04:51:31.963292+0000","last_peered":"2024-09-16T04:51:31.963292+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:31.963292+0000","last_undegraded":"2024-09-16T04:51:31.963292+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7545,"log_dups_size":0,"ondisk_log_size":7545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30879744,"num_objects":7539,"num_object_clones":0,"num_object_copies":15078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4268,"num_write_kb":17072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":7539}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7466","reported_seq":7554,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:32.034646+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:32.034646+0000","last_peered":"2024-09-16T04:51:32.034646+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:32.034646+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7466,"log_dups_size":0,"ondisk_log_size":7466,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30573161,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7465,"num_write_kb":29857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7723","reported_seq":7821,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:31.943239+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:31.943239+0000","last_peered":"2024-09-16T04:51:31.943239+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:31.943239+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7723,"log_dups_size":0,"ondisk_log_size":7723,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31602491,"num_objects":7718,"num_object_clones":0,"num_object_copies":15436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7720,"num_write_kb":30864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4204}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'7463","reported_seq":7590,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.613507+0000","last_change":"2024-09-16T04:49:38.182978+0000","last_active":"2024-09-16T04:51:27.613507+0000","last_peered":"2024-09-16T04:51:27.613507+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:27.613507+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7463,"log_dups_size":0,"ondisk_log_size":7463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30552064,"num_objects":7459,"num_object_clones":0,"num_object_copies":14918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4217,"num_write_kb":16868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4134},{"shards":"0,2","objects":3323}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'7572","reported_seq":7707,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:27.614460+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:27.614460+0000","last_peered":"2024-09-16T04:51:27.614460+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:27.614460+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":7572,"log_dups_size":0,"ondisk_log_size":7572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30966377,"num_objects":7561,"num_object_clones":0,"num_object_copies":15122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4341,"num_write_kb":17364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4037},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":491032799,"num_objects":119887,"num_object_clones":0,"num_object_copies":239774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37387,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85121,"num_write_kb":340456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3355,"num_bytes_recovered":13709312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":996323328,"data_stored":996243311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120043,"ondisk_log_size":120043,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596504,"kb_used_data":384328,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93775336,"statfs":{"total":96636764160,"available":96025944064,"internally_reserved":0,"allocated":393551872,"data_stored":393293110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":217247401},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,0,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593752,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376972,"kb_used_data":210620,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93994868,"statfs":{"total":96636764160,"available":96250744832,"internally_reserved":0,"allocated":215674880,"data_stored":215346538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170323630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,6,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580096,"kb_used_data":377712,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93791744,"statfs":{"total":96636764160,"available":96042745856,"internally_reserved":0,"allocated":386777088,"data_stored":386518326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":207220394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,1,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":387350528,"data_stored":387347049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":215678976,"data_stored":215605917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":393293824,"data_stored":393290345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:33.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:51:33.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'7296", 'reported_seq': 7371, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:27.493931+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:51:27.493931+0000', 'last_peered': '2024-09-16T04:51:27.493931+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:51:27.493931+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 7296, 'log_dups_size': 0, 'ondisk_log_size': 7296, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29884416, 'num_objects': 7296, 'num_object_clones': 0, 'num_object_copies': 14592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7296, 'num_write_kb': 29184, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3955}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:33.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:51:33.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'7498", 'reported_seq': 7632, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:31.910878+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:51:31.910878+0000', 'last_peered': '2024-09-16T04:51:31.910878+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:51:31.910878+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 7498, 'log_dups_size': 0, 'ondisk_log_size': 7498, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30695424, 'num_objects': 7494, 'num_object_clones': 0, 'num_object_copies': 14988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7494, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4237, 'num_write_kb': 16948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 4175}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:33.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:51:33.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'7487", 'reported_seq': 7616, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:27.607462+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:51:27.607462+0000', 'last_peered': '2024-09-16T04:51:27.607462+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:51:27.607462+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 7487, 'log_dups_size': 0, 'ondisk_log_size': 7487, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30646272, 'num_objects': 7482, 'num_object_clones': 0, 'num_object_copies': 14964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7482, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4480, 'num_write_kb': 17920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4026}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:33.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:51:33.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'7356", 'reported_seq': 7500, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:27.581206+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:51:27.581206+0000', 'last_peered': '2024-09-16T04:51:27.581206+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:51:27.581206+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 7356, 'log_dups_size': 0, 'ondisk_log_size': 7356, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30105600, 'num_objects': 7350, 'num_object_clones': 0, 'num_object_copies': 14700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7350, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4011, 'num_write_kb': 16044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3951}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:33.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:51:33.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'7591", 'reported_seq': 7814, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:31.924429+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:51:31.924429+0000', 'last_peered': '2024-09-16T04:51:31.924429+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:51:31.924429+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 7591, 'log_dups_size': 0, 'ondisk_log_size': 7591, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30957568, 'num_objects': 7558, 'num_object_clones': 0, 'num_object_copies': 15116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7558, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4342, 'num_write_kb': 17368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 4237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:33.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:51:33.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'7499", 'reported_seq': 7624, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:27.597337+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:51:27.597337+0000', 'last_peered': '2024-09-16T04:51:27.597337+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:51:27.597337+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 7499, 'log_dups_size': 0, 'ondisk_log_size': 7499, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30683136, 'num_objects': 7491, 'num_object_clones': 0, 'num_object_copies': 14982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7491, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4154, 'num_write_kb': 16616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4026}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:33.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:51:33.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'7494", 'reported_seq': 7732, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:31.987286+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:51:31.987286+0000', 'last_peered': '2024-09-16T04:51:31.987286+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:51:31.987286+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 7494, 'log_dups_size': 0, 'ondisk_log_size': 7494, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30569682, 'num_objects': 7465, 'num_object_clones': 0, 'num_object_copies': 14930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7465, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7466, 'num_write_kb': 29855, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 4169}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:33.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-16T04:51:33.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "45'7545", 'reported_seq': 14392, 'reported_epoch': 45, 'state': 'active+undersized+remapped', 'last_fresh': '2024-09-16T04:51:31.963292+0000', 'last_change': '2024-09-16T04:51:31.738291+0000', 'last_active': '2024-09-16T04:51:31.963292+0000', 'last_peered': '2024-09-16T04:51:31.963292+0000', 'last_clean': '2024-09-16T04:49:35.635641+0000', 'last_became_active': '2024-09-16T04:49:38.021871+0000', 'last_became_peered': '2024-09-16T04:49:38.021871+0000', 'last_unstale': '2024-09-16T04:51:31.963292+0000', 'last_undegraded': '2024-09-16T04:51:31.963292+0000', 'last_fullsized': '2024-09-16T04:49:37.644969+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3277", 'last_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.855144+0000', 'objects_scrubbed': 3271, 'log_size': 7545, 'log_dups_size': 0, 'ondisk_log_size': 7545, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 447, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30879744, 'num_objects': 7539, 'num_object_clones': 0, 'num_object_copies': 15078, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7539, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4268, 'num_write_kb': 17072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3346, 'num_bytes_recovered': 13697024, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0', '1'], 'object_location_counts': [{'shards': '2', 'objects': 7539}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:33.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:51:33.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'7466", 'reported_seq': 7554, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:32.034646+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:51:32.034646+0000', 'last_peered': '2024-09-16T04:51:32.034646+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:51:32.034646+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 7466, 'log_dups_size': 0, 'ondisk_log_size': 7466, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30573161, 'num_objects': 7465, 'num_object_clones': 0, 'num_object_copies': 14930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7465, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7465, 'num_write_kb': 29857, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 4058}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:33.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:51:33.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'7723", 'reported_seq': 7821, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:31.943239+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:51:31.943239+0000', 'last_peered': '2024-09-16T04:51:31.943239+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:51:31.943239+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 7723, 'log_dups_size': 0, 'ondisk_log_size': 7723, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31602491, 'num_objects': 7718, 'num_object_clones': 0, 'num_object_copies': 15436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7720, 'num_write_kb': 30864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 4204}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:33.046 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:51:33.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'7463", 'reported_seq': 7590, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:27.613507+0000', 'last_change': '2024-09-16T04:49:38.182978+0000', 'last_active': '2024-09-16T04:51:27.613507+0000', 'last_peered': '2024-09-16T04:51:27.613507+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:51:27.613507+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 7463, 'log_dups_size': 0, 'ondisk_log_size': 7463, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30552064, 'num_objects': 7459, 'num_object_clones': 0, 'num_object_copies': 14918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7459, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4217, 'num_write_kb': 16868, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4134}, {'shards': '0,2', 'objects': 3323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:33.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:51:33.047 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'7572", 'reported_seq': 7707, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:27.614460+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:51:27.614460+0000', 'last_peered': '2024-09-16T04:51:27.614460+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:51:27.614460+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 7572, 'log_dups_size': 0, 'ondisk_log_size': 7572, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30966377, 'num_objects': 7561, 'num_object_clones': 0, 'num_object_copies': 15122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7561, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4341, 'num_write_kb': 17364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4037}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:33.047 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:51:33.094 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 156 20 121429 121409 3.03914 1.71484 0.0211248 0.0256284 2024-09-16T04:51:33.873 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:33.873 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37387/239778 objects degraded (15.592%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":241,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":119889,"data_bytes":491032818,"bytes_used":1590857728,"bytes_avail":288319434752,"bytes_total":289910292480,"degraded_objects":37387,"degraded_total":239778,"degraded_ratio":0.15592339580778888,"recovering_objects_per_sec":48,"recovering_bytes_per_sec":199018,"recovering_keys_per_sec":0,"num_objects_recovered":486,"num_bytes_recovered":1990656,"num_keys_recovered":0,"write_bytes_sec":2115087,"read_op_per_sec":0,"write_op_per_sec":516},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:51:33.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:51:33.940 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:34.100 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 157 20 121896 121876 3.03135 1.82422 0.0497174 0.0257652 2024-09-16T04:51:34.832 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:34.833 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:34.893 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-16T04:51:34.319847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":496111858,"num_objects":121129,"num_object_clones":0,"num_object_copies":242258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3300,"num_objects_unfound":0,"num_objects_dirty":121129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86363,"num_write_kb":345418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3379,"num_bytes_recovered":13803539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121286,"ondisk_log_size":121286,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1547156,"kb_used_data":978724,"kb_used_omap":12,"kb_used_meta":568371,"kb_avail":281568364,"statfs":{"total":289910292480,"available":288326004736,"internally_reserved":0,"allocated":1002213376,"data_stored":1001367510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":582011905},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,5,4,13,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20873216,"num_objects":5096,"num_object_clones":0,"num_object_copies":10192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3325,"num_objects_misplaced":2814,"num_objects_unfound":0,"num_objects_dirty":5096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5096,"num_write_kb":20384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":2084864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001890"},"pg_stats":[{"pgid":"2.f","version":"46'7438","reported_seq":7515,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:32.585887+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:32.585887+0000","last_peered":"2024-09-16T04:51:32.585887+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:32.585887+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7438,"log_dups_size":0,"ondisk_log_size":7438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30466048,"num_objects":7438,"num_object_clones":0,"num_object_copies":14876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7438,"num_write_kb":29752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4097},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'7537","reported_seq":7649,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:31.956472+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:31.956472+0000","last_peered":"2024-09-16T04:51:31.956472+0000","last_clean":"2024-09-16T04:51:31.956472+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:31.956472+0000","last_undegraded":"2024-09-16T04:51:31.956472+0000","last_fullsized":"2024-09-16T04:51:31.956472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7537,"log_dups_size":0,"ondisk_log_size":7537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30846976,"num_objects":7531,"num_object_clones":0,"num_object_copies":15062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4506,"num_write_kb":18024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'7498","reported_seq":7632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:31.910878+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:31.910878+0000","last_peered":"2024-09-16T04:51:31.910878+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:31.910878+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7498,"log_dups_size":0,"ondisk_log_size":7498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30695424,"num_objects":7494,"num_object_clones":0,"num_object_copies":14988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4237,"num_write_kb":16948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4175}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'7629","reported_seq":7695,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:51:31.997617+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:31.997617+0000","last_peered":"2024-09-16T04:51:31.997617+0000","last_clean":"2024-09-16T04:51:31.997617+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:31.997617+0000","last_undegraded":"2024-09-16T04:51:31.997617+0000","last_fullsized":"2024-09-16T04:51:31.997617+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7629,"log_dups_size":0,"ondisk_log_size":7629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31248384,"num_objects":7629,"num_object_clones":0,"num_object_copies":15258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7629,"num_write_kb":30516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"46'7638","reported_seq":7854,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T04:51:32.652159+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:32.652159+0000","last_peered":"2024-09-16T04:51:32.652159+0000","last_clean":"2024-09-16T04:51:32.652159+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:32.652159+0000","last_undegraded":"2024-09-16T04:51:32.652159+0000","last_fullsized":"2024-09-16T04:51:32.652159+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7638,"log_dups_size":0,"ondisk_log_size":7638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162368,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4670,"num_write_kb":18680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"46'7660","reported_seq":7791,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:32.628537+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:32.628537+0000","last_peered":"2024-09-16T04:51:32.628537+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:32.628537+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7660,"log_dups_size":0,"ondisk_log_size":7660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31354880,"num_objects":7655,"num_object_clones":0,"num_object_copies":15310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4653,"num_write_kb":18612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4199},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"46'7514","reported_seq":7660,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:32.647557+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:32.647557+0000","last_peered":"2024-09-16T04:51:32.647557+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:32.647557+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7514,"log_dups_size":0,"ondisk_log_size":7514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30752768,"num_objects":7508,"num_object_clones":0,"num_object_copies":15016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4169,"num_write_kb":16676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4109},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"46'7553","reported_seq":7766,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T04:51:32.637360+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:32.637360+0000","last_peered":"2024-09-16T04:51:32.637360+0000","last_clean":"2024-09-16T04:51:32.637360+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:32.637360+0000","last_undegraded":"2024-09-16T04:51:32.637360+0000","last_fullsized":"2024-09-16T04:51:32.637360+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7553,"log_dups_size":0,"ondisk_log_size":7553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30904320,"num_objects":7545,"num_object_clones":0,"num_object_copies":15090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4623,"num_write_kb":18492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.795980+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:49:37.795980+0000","last_peered":"2024-09-16T04:49:37.795980+0000","last_clean":"2024-09-16T04:49:37.795980+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:49:37.795980+0000","last_undegraded":"2024-09-16T04:49:37.795980+0000","last_fullsized":"2024-09-16T04:49:37.795980+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'7591","reported_seq":7814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:31.924429+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:31.924429+0000","last_peered":"2024-09-16T04:51:31.924429+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:31.924429+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7591,"log_dups_size":0,"ondisk_log_size":7591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30957568,"num_objects":7558,"num_object_clones":0,"num_object_copies":15116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4342,"num_write_kb":17368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":4237}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.783920+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:49:37.783920+0000","last_peered":"2024-09-16T04:49:37.783920+0000","last_clean":"2024-09-16T04:49:37.783920+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:49:37.783920+0000","last_undegraded":"2024-09-16T04:49:37.783920+0000","last_fullsized":"2024-09-16T04:49:37.783920+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"46'7644","reported_seq":7771,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:32.667251+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:32.667251+0000","last_peered":"2024-09-16T04:51:32.667251+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:32.667251+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7644,"log_dups_size":0,"ondisk_log_size":7644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31277056,"num_objects":7636,"num_object_clones":0,"num_object_copies":15272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4299,"num_write_kb":17196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4171},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T04:51:32.163915+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:32.163915+0000","last_peered":"2024-09-16T04:51:32.163915+0000","last_clean":"2024-09-16T04:51:32.163915+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:32.163915+0000","last_undegraded":"2024-09-16T04:51:32.163915+0000","last_fullsized":"2024-09-16T04:51:32.163915+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'7494","reported_seq":7732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:31.987286+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:31.987286+0000","last_peered":"2024-09-16T04:51:31.987286+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:31.987286+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7494,"log_dups_size":0,"ondisk_log_size":7494,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30569682,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7466,"num_write_kb":29855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4169}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.841481+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:49:37.841481+0000","last_peered":"2024-09-16T04:49:37.841481+0000","last_clean":"2024-09-16T04:49:37.841481+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:49:37.841481+0000","last_undegraded":"2024-09-16T04:49:37.841481+0000","last_fullsized":"2024-09-16T04:49:37.841481+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'7545","reported_seq":14392,"reported_epoch":45,"state":"active+undersized+remapped","last_fresh":"2024-09-16T04:51:31.963292+0000","last_change":"2024-09-16T04:51:31.738291+0000","last_active":"2024-09-16T04:51:31.963292+0000","last_peered":"2024-09-16T04:51:31.963292+0000","last_clean":"2024-09-16T04:49:35.635641+0000","last_became_active":"2024-09-16T04:49:38.021871+0000","last_became_peered":"2024-09-16T04:49:38.021871+0000","last_unstale":"2024-09-16T04:51:31.963292+0000","last_undegraded":"2024-09-16T04:51:31.963292+0000","last_fullsized":"2024-09-16T04:49:37.644969+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7545,"log_dups_size":0,"ondisk_log_size":7545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30879744,"num_objects":7539,"num_object_clones":0,"num_object_copies":15078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4268,"num_write_kb":17072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0","1"],"object_location_counts":[{"shards":"2","objects":7539}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'7466","reported_seq":7554,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:32.034646+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:32.034646+0000","last_peered":"2024-09-16T04:51:32.034646+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:32.034646+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7466,"log_dups_size":0,"ondisk_log_size":7466,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30573161,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7465,"num_write_kb":29857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.837167+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:49:37.837167+0000","last_peered":"2024-09-16T04:49:37.837167+0000","last_clean":"2024-09-16T04:49:37.837167+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:49:37.837167+0000","last_undegraded":"2024-09-16T04:49:37.837167+0000","last_fullsized":"2024-09-16T04:49:37.837167+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'7723","reported_seq":7821,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:31.943239+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:31.943239+0000","last_peered":"2024-09-16T04:51:31.943239+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:31.943239+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7723,"log_dups_size":0,"ondisk_log_size":7723,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31602491,"num_objects":7718,"num_object_clones":0,"num_object_copies":15436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7720,"num_write_kb":30864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4204}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.781419+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:49:37.781419+0000","last_peered":"2024-09-16T04:49:37.781419+0000","last_clean":"2024-09-16T04:49:37.781419+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:49:37.781419+0000","last_undegraded":"2024-09-16T04:49:37.781419+0000","last_fullsized":"2024-09-16T04:49:37.781419+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"46'7621","reported_seq":7797,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:32.667179+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:51:32.667179+0000","last_peered":"2024-09-16T04:51:32.667179+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:32.667179+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7621,"log_dups_size":0,"ondisk_log_size":7621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31199232,"num_objects":7617,"num_object_clones":0,"num_object_copies":15234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3300,"num_objects_unfound":0,"num_objects_dirty":7617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4375,"num_write_kb":17500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":94208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4315},{"shards":"0,2","objects":3300}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:37.663717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:49:37.663717+0000","last_peered":"2024-09-16T04:49:37.663717+0000","last_clean":"2024-09-16T04:49:37.663717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:49:37.663717+0000","last_undegraded":"2024-09-16T04:49:37.663717+0000","last_fullsized":"2024-09-16T04:49:37.663717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T04:49:38.136373+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:49:38.136373+0000","last_peered":"2024-09-16T04:49:38.136373+0000","last_clean":"2024-09-16T04:49:38.136373+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:49:38.136373+0000","last_undegraded":"2024-09-16T04:49:38.136373+0000","last_fullsized":"2024-09-16T04:49:38.136373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"46'7732","reported_seq":7867,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:32.659778+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:32.659778+0000","last_peered":"2024-09-16T04:51:32.659778+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:32.659778+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":7732,"log_dups_size":0,"ondisk_log_size":7732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31621737,"num_objects":7721,"num_object_clones":0,"num_object_copies":15442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4501,"num_write_kb":18004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4197},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":496111839,"num_objects":121127,"num_object_clones":0,"num_object_copies":242254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3300,"num_objects_unfound":0,"num_objects_dirty":121127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86361,"num_write_kb":345416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3378,"num_bytes_recovered":13803520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1002725376,"data_stored":1002645359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121283,"ondisk_log_size":121283,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596504,"kb_used_data":384328,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93775336,"statfs":{"total":96636764160,"available":96025944064,"internally_reserved":0,"allocated":393551872,"data_stored":393293110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":217247401},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,0,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593752,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376972,"kb_used_data":210620,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93994868,"statfs":{"total":96636764160,"available":96250744832,"internally_reserved":0,"allocated":215674880,"data_stored":215346538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170323630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,6,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573680,"kb_used_data":383776,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93798160,"statfs":{"total":96636764160,"available":96049315840,"internally_reserved":0,"allocated":392986624,"data_stored":392727862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":194440874},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":393752576,"data_stored":393749097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":215678976,"data_stored":215605917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":393293824,"data_stored":393290345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:35.097 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 158 20 122384 122364 3.02423 1.90625 0.00970636 0.0258227 2024-09-16T04:51:36.102 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 159 20 122907 122887 3.018 2.04297 0.0491554 0.0258786 2024-09-16T04:51:37.100 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:2024-09-16T04:51:37.099744+0000 min lat: 0.00206892 max lat: 2.69414 avg lat: 0.0259307 2024-09-16T04:51:37.101 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T04:51:37.101 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 160 20 123439 123419 3.01213 2.07812 0.0336282 0.0259307 2024-09-16T04:51:37.895 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:38.101 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 161 20 123949 123929 3.00578 1.99219 0.0232765 0.0259841 2024-09-16T04:51:38.646 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:38.646 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:38.702 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-16T04:51:38.320861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506097906,"num_objects":123567,"num_object_clones":0,"num_object_copies":247134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3210,"num_objects_unfound":0,"num_objects_dirty":123567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88801,"num_write_kb":355170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3469,"num_bytes_recovered":14172179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123724,"ondisk_log_size":123724,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1715752,"kb_used_data":998768,"kb_used_omap":12,"kb_used_meta":716915,"kb_avail":281399768,"statfs":{"total":289910292480,"available":288153362432,"internally_reserved":0,"allocated":1022738432,"data_stored":1021861903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":734120961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,4,5,7,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20410368,"num_objects":4983,"num_object_clones":0,"num_object_copies":9966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3325,"num_objects_misplaced":2968,"num_objects_unfound":0,"num_objects_dirty":4983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4983,"num_write_kb":19932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":1454080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001905"},"pg_stats":[{"pgid":"2.f","version":"47'7596","reported_seq":7672,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.743615+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:37.743615+0000","last_peered":"2024-09-16T04:51:37.743615+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:37.743615+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7596,"log_dups_size":0,"ondisk_log_size":7596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31113216,"num_objects":7596,"num_object_clones":0,"num_object_copies":15192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7596,"num_write_kb":30384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'7693","reported_seq":7809,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:36.943531+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:36.943531+0000","last_peered":"2024-09-16T04:51:36.943531+0000","last_clean":"2024-09-16T04:51:36.943531+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:36.943531+0000","last_undegraded":"2024-09-16T04:51:36.943531+0000","last_fullsized":"2024-09-16T04:51:36.943531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7693,"log_dups_size":0,"ondisk_log_size":7693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31485952,"num_objects":7687,"num_object_clones":0,"num_object_copies":15374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4662,"num_write_kb":18648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'7637","reported_seq":7775,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.014782+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:37.014782+0000","last_peered":"2024-09-16T04:51:37.014782+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:37.014782+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7637,"log_dups_size":0,"ondisk_log_size":7637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31264768,"num_objects":7633,"num_object_clones":0,"num_object_copies":15266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4376,"num_write_kb":17504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4314}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'7780","reported_seq":7850,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:36.986184+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:36.986184+0000","last_peered":"2024-09-16T04:51:36.986184+0000","last_clean":"2024-09-16T04:51:36.986184+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:36.986184+0000","last_undegraded":"2024-09-16T04:51:36.986184+0000","last_fullsized":"2024-09-16T04:51:36.986184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7780,"log_dups_size":0,"ondisk_log_size":7780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31866880,"num_objects":7780,"num_object_clones":0,"num_object_copies":15560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7780,"num_write_kb":31120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'7792","reported_seq":8010,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:37.623882+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:37.623882+0000","last_peered":"2024-09-16T04:51:37.623882+0000","last_clean":"2024-09-16T04:51:37.623882+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:37.623882+0000","last_undegraded":"2024-09-16T04:51:37.623882+0000","last_fullsized":"2024-09-16T04:51:37.623882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7792,"log_dups_size":0,"ondisk_log_size":7792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31793152,"num_objects":7762,"num_object_clones":0,"num_object_copies":15524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4824,"num_write_kb":19296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'7815","reported_seq":7948,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.638148+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:37.638148+0000","last_peered":"2024-09-16T04:51:37.638148+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:37.638148+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7815,"log_dups_size":0,"ondisk_log_size":7815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31989760,"num_objects":7810,"num_object_clones":0,"num_object_copies":15620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4808,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4354},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'7686","reported_seq":7833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.626083+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:37.626083+0000","last_peered":"2024-09-16T04:51:37.626083+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:37.626083+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7686,"log_dups_size":0,"ondisk_log_size":7686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31457280,"num_objects":7680,"num_object_clones":0,"num_object_copies":15360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4341,"num_write_kb":17364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'7713","reported_seq":7929,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:37.694365+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:37.694365+0000","last_peered":"2024-09-16T04:51:37.694365+0000","last_clean":"2024-09-16T04:51:37.694365+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:37.694365+0000","last_undegraded":"2024-09-16T04:51:37.694365+0000","last_fullsized":"2024-09-16T04:51:37.694365+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7713,"log_dups_size":0,"ondisk_log_size":7713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31559680,"num_objects":7705,"num_object_clones":0,"num_object_copies":15410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4783,"num_write_kb":19132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'7745","reported_seq":7973,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:36.991629+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:36.991629+0000","last_peered":"2024-09-16T04:51:36.991629+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:36.991629+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7745,"log_dups_size":0,"ondisk_log_size":7745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31588352,"num_objects":7712,"num_object_clones":0,"num_object_copies":15424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4496,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":4391}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'7793","reported_seq":7919,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.707458+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:37.707458+0000","last_peered":"2024-09-16T04:51:37.707458+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:37.707458+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7793,"log_dups_size":0,"ondisk_log_size":7793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31887360,"num_objects":7785,"num_object_clones":0,"num_object_copies":15570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4448,"num_write_kb":17792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4320},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'7637","reported_seq":7879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.017380+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:37.017380+0000","last_peered":"2024-09-16T04:51:37.017380+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:37.017380+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7637,"log_dups_size":0,"ondisk_log_size":7637,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31155410,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7609,"num_write_kb":30427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4312}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":73,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149422+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149422+0000","last_peered":"2024-09-16T04:51:33.149422+0000","last_clean":"2024-09-16T04:51:33.149422+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149422+0000","last_undegraded":"2024-09-16T04:51:33.149422+0000","last_fullsized":"2024-09-16T04:51:33.149422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'7694","reported_seq":14550,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:37.005085+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:51:37.005085+0000","last_peered":"2024-09-16T04:51:37.005085+0000","last_clean":"2024-09-16T04:51:37.005085+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:51:37.005085+0000","last_undegraded":"2024-09-16T04:51:37.005085+0000","last_fullsized":"2024-09-16T04:51:37.005085+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7694,"log_dups_size":0,"ondisk_log_size":7694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31490048,"num_objects":7688,"num_object_clones":0,"num_object_copies":15376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4417,"num_write_kb":17668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'7611","reported_seq":7708,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:36.964379+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:36.964379+0000","last_peered":"2024-09-16T04:51:36.964379+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:36.964379+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7611,"log_dups_size":0,"ondisk_log_size":7611,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31167081,"num_objects":7610,"num_object_clones":0,"num_object_copies":15220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7610,"num_write_kb":30437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149652+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149652+0000","last_peered":"2024-09-16T04:51:33.149652+0000","last_clean":"2024-09-16T04:51:33.149652+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149652+0000","last_undegraded":"2024-09-16T04:51:33.149652+0000","last_fullsized":"2024-09-16T04:51:33.149652+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'7878","reported_seq":7980,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.019635+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:37.019635+0000","last_peered":"2024-09-16T04:51:37.019635+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:37.019635+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7878,"log_dups_size":0,"ondisk_log_size":7878,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32237371,"num_objects":7873,"num_object_clones":0,"num_object_copies":15746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7875,"num_write_kb":31484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4359}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'7786","reported_seq":8144,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.712314+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:51:37.712314+0000","last_peered":"2024-09-16T04:51:37.712314+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:37.712314+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7786,"log_dups_size":0,"ondisk_log_size":7786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875072,"num_objects":7782,"num_object_clones":0,"num_object_copies":15564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3210,"num_objects_unfound":0,"num_objects_dirty":7782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4540,"num_write_kb":18160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":462848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4570},{"shards":"0,2","objects":3210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150640+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150640+0000","last_peered":"2024-09-16T04:51:33.150640+0000","last_clean":"2024-09-16T04:51:33.150640+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150640+0000","last_undegraded":"2024-09-16T04:51:33.150640+0000","last_fullsized":"2024-09-16T04:51:33.150640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155436+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155436+0000","last_peered":"2024-09-16T04:51:33.155436+0000","last_clean":"2024-09-16T04:51:33.155436+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155436+0000","last_undegraded":"2024-09-16T04:51:33.155436+0000","last_fullsized":"2024-09-16T04:51:33.155436+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'7865","reported_seq":8005,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.630008+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:37.630008+0000","last_peered":"2024-09-16T04:51:37.630008+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:37.630008+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":7865,"log_dups_size":0,"ondisk_log_size":7865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32166505,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4634,"num_write_kb":18536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4330},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":506097887,"num_objects":123565,"num_object_clones":0,"num_object_copies":247130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3210,"num_objects_unfound":0,"num_objects_dirty":123565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88799,"num_write_kb":355168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3468,"num_bytes_recovered":14172160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1023385600,"data_stored":1023305583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123721,"ondisk_log_size":123721,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664856,"kb_used_data":390212,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93706984,"statfs":{"total":96636764160,"available":95955951616,"internally_reserved":0,"allocated":399577088,"data_stored":399308105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,3,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593753,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394644,"kb_used_data":218560,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93977196,"statfs":{"total":96636764160,"available":96232648704,"internally_reserved":0,"allocated":223805440,"data_stored":223466877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":180285102},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656252,"kb_used_data":389996,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93715588,"statfs":{"total":96636764160,"available":95964762112,"internally_reserved":0,"allocated":399355904,"data_stored":399086921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":400175104,"data_stored":400171625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":223719424,"data_stored":223646365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":399491072,"data_stored":399487593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:38.704 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:39.101 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 162 20 124431 124411 2.99885 1.88281 0.0579818 0.0260446 2024-09-16T04:51:39.570 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:39.570 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:39.635 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-16T04:51:38.320861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506097906,"num_objects":123567,"num_object_clones":0,"num_object_copies":247134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3210,"num_objects_unfound":0,"num_objects_dirty":123567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88801,"num_write_kb":355170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3469,"num_bytes_recovered":14172179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123724,"ondisk_log_size":123724,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1715752,"kb_used_data":998768,"kb_used_omap":12,"kb_used_meta":716915,"kb_avail":281399768,"statfs":{"total":289910292480,"available":288153362432,"internally_reserved":0,"allocated":1022738432,"data_stored":1021861903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":734120961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,4,5,7,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20410368,"num_objects":4983,"num_object_clones":0,"num_object_copies":9966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3325,"num_objects_misplaced":2968,"num_objects_unfound":0,"num_objects_dirty":4983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4983,"num_write_kb":19932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":1454080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001905"},"pg_stats":[{"pgid":"2.f","version":"47'7596","reported_seq":7672,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.743615+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:37.743615+0000","last_peered":"2024-09-16T04:51:37.743615+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:37.743615+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7596,"log_dups_size":0,"ondisk_log_size":7596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31113216,"num_objects":7596,"num_object_clones":0,"num_object_copies":15192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7596,"num_write_kb":30384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'7693","reported_seq":7809,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:36.943531+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:36.943531+0000","last_peered":"2024-09-16T04:51:36.943531+0000","last_clean":"2024-09-16T04:51:36.943531+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:36.943531+0000","last_undegraded":"2024-09-16T04:51:36.943531+0000","last_fullsized":"2024-09-16T04:51:36.943531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7693,"log_dups_size":0,"ondisk_log_size":7693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31485952,"num_objects":7687,"num_object_clones":0,"num_object_copies":15374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4662,"num_write_kb":18648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'7637","reported_seq":7775,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.014782+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:37.014782+0000","last_peered":"2024-09-16T04:51:37.014782+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:37.014782+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7637,"log_dups_size":0,"ondisk_log_size":7637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31264768,"num_objects":7633,"num_object_clones":0,"num_object_copies":15266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4376,"num_write_kb":17504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4314}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'7780","reported_seq":7850,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:36.986184+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:36.986184+0000","last_peered":"2024-09-16T04:51:36.986184+0000","last_clean":"2024-09-16T04:51:36.986184+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:36.986184+0000","last_undegraded":"2024-09-16T04:51:36.986184+0000","last_fullsized":"2024-09-16T04:51:36.986184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7780,"log_dups_size":0,"ondisk_log_size":7780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31866880,"num_objects":7780,"num_object_clones":0,"num_object_copies":15560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7780,"num_write_kb":31120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'7792","reported_seq":8010,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:37.623882+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:37.623882+0000","last_peered":"2024-09-16T04:51:37.623882+0000","last_clean":"2024-09-16T04:51:37.623882+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:37.623882+0000","last_undegraded":"2024-09-16T04:51:37.623882+0000","last_fullsized":"2024-09-16T04:51:37.623882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7792,"log_dups_size":0,"ondisk_log_size":7792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31793152,"num_objects":7762,"num_object_clones":0,"num_object_copies":15524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4824,"num_write_kb":19296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'7815","reported_seq":7948,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.638148+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:37.638148+0000","last_peered":"2024-09-16T04:51:37.638148+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:37.638148+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7815,"log_dups_size":0,"ondisk_log_size":7815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31989760,"num_objects":7810,"num_object_clones":0,"num_object_copies":15620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4808,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4354},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'7686","reported_seq":7833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.626083+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:37.626083+0000","last_peered":"2024-09-16T04:51:37.626083+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:37.626083+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7686,"log_dups_size":0,"ondisk_log_size":7686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31457280,"num_objects":7680,"num_object_clones":0,"num_object_copies":15360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4341,"num_write_kb":17364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'7713","reported_seq":7929,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:37.694365+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:37.694365+0000","last_peered":"2024-09-16T04:51:37.694365+0000","last_clean":"2024-09-16T04:51:37.694365+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:37.694365+0000","last_undegraded":"2024-09-16T04:51:37.694365+0000","last_fullsized":"2024-09-16T04:51:37.694365+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7713,"log_dups_size":0,"ondisk_log_size":7713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31559680,"num_objects":7705,"num_object_clones":0,"num_object_copies":15410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4783,"num_write_kb":19132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'7745","reported_seq":7973,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:36.991629+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:36.991629+0000","last_peered":"2024-09-16T04:51:36.991629+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:36.991629+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7745,"log_dups_size":0,"ondisk_log_size":7745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31588352,"num_objects":7712,"num_object_clones":0,"num_object_copies":15424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4496,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":4391}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'7793","reported_seq":7919,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.707458+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:37.707458+0000","last_peered":"2024-09-16T04:51:37.707458+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:37.707458+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7793,"log_dups_size":0,"ondisk_log_size":7793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31887360,"num_objects":7785,"num_object_clones":0,"num_object_copies":15570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4448,"num_write_kb":17792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4320},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'7637","reported_seq":7879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.017380+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:37.017380+0000","last_peered":"2024-09-16T04:51:37.017380+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:37.017380+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7637,"log_dups_size":0,"ondisk_log_size":7637,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31155410,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7609,"num_write_kb":30427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4312}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":73,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149422+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149422+0000","last_peered":"2024-09-16T04:51:33.149422+0000","last_clean":"2024-09-16T04:51:33.149422+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149422+0000","last_undegraded":"2024-09-16T04:51:33.149422+0000","last_fullsized":"2024-09-16T04:51:33.149422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'7694","reported_seq":14550,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:37.005085+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:51:37.005085+0000","last_peered":"2024-09-16T04:51:37.005085+0000","last_clean":"2024-09-16T04:51:37.005085+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:51:37.005085+0000","last_undegraded":"2024-09-16T04:51:37.005085+0000","last_fullsized":"2024-09-16T04:51:37.005085+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7694,"log_dups_size":0,"ondisk_log_size":7694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31490048,"num_objects":7688,"num_object_clones":0,"num_object_copies":15376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4417,"num_write_kb":17668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'7611","reported_seq":7708,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:36.964379+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:36.964379+0000","last_peered":"2024-09-16T04:51:36.964379+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:36.964379+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7611,"log_dups_size":0,"ondisk_log_size":7611,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31167081,"num_objects":7610,"num_object_clones":0,"num_object_copies":15220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7610,"num_write_kb":30437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149652+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149652+0000","last_peered":"2024-09-16T04:51:33.149652+0000","last_clean":"2024-09-16T04:51:33.149652+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149652+0000","last_undegraded":"2024-09-16T04:51:33.149652+0000","last_fullsized":"2024-09-16T04:51:33.149652+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'7878","reported_seq":7980,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.019635+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:37.019635+0000","last_peered":"2024-09-16T04:51:37.019635+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:37.019635+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7878,"log_dups_size":0,"ondisk_log_size":7878,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32237371,"num_objects":7873,"num_object_clones":0,"num_object_copies":15746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7875,"num_write_kb":31484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4359}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'7786","reported_seq":8144,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.712314+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:51:37.712314+0000","last_peered":"2024-09-16T04:51:37.712314+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:37.712314+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7786,"log_dups_size":0,"ondisk_log_size":7786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875072,"num_objects":7782,"num_object_clones":0,"num_object_copies":15564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3210,"num_objects_unfound":0,"num_objects_dirty":7782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4540,"num_write_kb":18160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":462848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4570},{"shards":"0,2","objects":3210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150640+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150640+0000","last_peered":"2024-09-16T04:51:33.150640+0000","last_clean":"2024-09-16T04:51:33.150640+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150640+0000","last_undegraded":"2024-09-16T04:51:33.150640+0000","last_fullsized":"2024-09-16T04:51:33.150640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155436+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155436+0000","last_peered":"2024-09-16T04:51:33.155436+0000","last_clean":"2024-09-16T04:51:33.155436+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155436+0000","last_undegraded":"2024-09-16T04:51:33.155436+0000","last_fullsized":"2024-09-16T04:51:33.155436+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'7865","reported_seq":8005,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.630008+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:37.630008+0000","last_peered":"2024-09-16T04:51:37.630008+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:37.630008+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":7865,"log_dups_size":0,"ondisk_log_size":7865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32166505,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4634,"num_write_kb":18536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4330},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":506097887,"num_objects":123565,"num_object_clones":0,"num_object_copies":247130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3210,"num_objects_unfound":0,"num_objects_dirty":123565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88799,"num_write_kb":355168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3468,"num_bytes_recovered":14172160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1023385600,"data_stored":1023305583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123721,"ondisk_log_size":123721,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664856,"kb_used_data":390212,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93706984,"statfs":{"total":96636764160,"available":95955951616,"internally_reserved":0,"allocated":399577088,"data_stored":399308105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,3,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593753,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394644,"kb_used_data":218560,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93977196,"statfs":{"total":96636764160,"available":96232648704,"internally_reserved":0,"allocated":223805440,"data_stored":223466877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":180285102},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656252,"kb_used_data":389996,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93715588,"statfs":{"total":96636764160,"available":95964762112,"internally_reserved":0,"allocated":399355904,"data_stored":399086921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":400175104,"data_stored":400171625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":223719424,"data_stored":223646365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":399491072,"data_stored":399487593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:39.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:51:39.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'7596", 'reported_seq': 7672, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:37.743615+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:51:37.743615+0000', 'last_peered': '2024-09-16T04:51:37.743615+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:51:37.743615+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 7596, 'log_dups_size': 0, 'ondisk_log_size': 7596, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31113216, 'num_objects': 7596, 'num_object_clones': 0, 'num_object_copies': 15192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7596, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7596, 'num_write_kb': 30384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4255}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:39.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:51:39.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'7637", 'reported_seq': 7775, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:37.014782+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:51:37.014782+0000', 'last_peered': '2024-09-16T04:51:37.014782+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:51:37.014782+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 7637, 'log_dups_size': 0, 'ondisk_log_size': 7637, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31264768, 'num_objects': 7633, 'num_object_clones': 0, 'num_object_copies': 15266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7633, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4376, 'num_write_kb': 17504, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 4314}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:39.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:51:39.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'7815", 'reported_seq': 7948, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:37.638148+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:51:37.638148+0000', 'last_peered': '2024-09-16T04:51:37.638148+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:51:37.638148+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 7815, 'log_dups_size': 0, 'ondisk_log_size': 7815, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31989760, 'num_objects': 7810, 'num_object_clones': 0, 'num_object_copies': 15620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7810, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4808, 'num_write_kb': 19232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4354}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:39.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:51:39.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'7686", 'reported_seq': 7833, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:37.626083+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:51:37.626083+0000', 'last_peered': '2024-09-16T04:51:37.626083+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:51:37.626083+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 7686, 'log_dups_size': 0, 'ondisk_log_size': 7686, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31457280, 'num_objects': 7680, 'num_object_clones': 0, 'num_object_copies': 15360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7680, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4341, 'num_write_kb': 17364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4281}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:39.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:51:39.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'7745", 'reported_seq': 7973, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:36.991629+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:51:36.991629+0000', 'last_peered': '2024-09-16T04:51:36.991629+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:51:36.991629+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 7745, 'log_dups_size': 0, 'ondisk_log_size': 7745, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31588352, 'num_objects': 7712, 'num_object_clones': 0, 'num_object_copies': 15424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7712, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4496, 'num_write_kb': 17984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 4391}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:39.639 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:51:39.639 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'7793", 'reported_seq': 7919, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:37.707458+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:51:37.707458+0000', 'last_peered': '2024-09-16T04:51:37.707458+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:51:37.707458+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 7793, 'log_dups_size': 0, 'ondisk_log_size': 7793, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31887360, 'num_objects': 7785, 'num_object_clones': 0, 'num_object_copies': 15570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7785, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4448, 'num_write_kb': 17792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4320}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:39.639 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:51:39.639 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'7637", 'reported_seq': 7879, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:37.017380+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:51:37.017380+0000', 'last_peered': '2024-09-16T04:51:37.017380+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:51:37.017380+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 7637, 'log_dups_size': 0, 'ondisk_log_size': 7637, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31155410, 'num_objects': 7608, 'num_object_clones': 0, 'num_object_copies': 15216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7608, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7609, 'num_write_kb': 30427, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 4312}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:39.639 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:51:39.639 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'7611", 'reported_seq': 7708, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:36.964379+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:51:36.964379+0000', 'last_peered': '2024-09-16T04:51:36.964379+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:51:36.964379+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 7611, 'log_dups_size': 0, 'ondisk_log_size': 7611, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31167081, 'num_objects': 7610, 'num_object_clones': 0, 'num_object_copies': 15220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7610, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7610, 'num_write_kb': 30437, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 4203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:39.639 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:51:39.639 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'7878", 'reported_seq': 7980, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:37.019635+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:51:37.019635+0000', 'last_peered': '2024-09-16T04:51:37.019635+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:51:37.019635+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 7878, 'log_dups_size': 0, 'ondisk_log_size': 7878, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32237371, 'num_objects': 7873, 'num_object_clones': 0, 'num_object_copies': 15746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7873, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7875, 'num_write_kb': 31484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 4359}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:39.639 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:51:39.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'7786", 'reported_seq': 8144, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:37.712314+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:51:37.712314+0000', 'last_peered': '2024-09-16T04:51:37.712314+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:51:37.712314+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 7786, 'log_dups_size': 0, 'ondisk_log_size': 7786, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31875072, 'num_objects': 7782, 'num_object_clones': 0, 'num_object_copies': 15564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3210, 'num_objects_unfound': 0, 'num_objects_dirty': 7782, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4540, 'num_write_kb': 18160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 113, 'num_bytes_recovered': 462848, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4570}, {'shards': '0,2', 'objects': 3210}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:39.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:51:39.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'7865", 'reported_seq': 8005, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:37.630008+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:51:37.630008+0000', 'last_peered': '2024-09-16T04:51:37.630008+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:51:37.630008+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 7865, 'log_dups_size': 0, 'ondisk_log_size': 7865, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32166505, 'num_objects': 7854, 'num_object_clones': 0, 'num_object_copies': 15708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7854, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4634, 'num_write_kb': 18536, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4330}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:39.640 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:51:40.101 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 163 20 124920 124900 2.99218 1.91016 0.0338928 0.0261015 2024-09-16T04:51:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:40.762 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/247134 objects degraded (13.783%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":248,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":123567,"data_bytes":506097906,"bytes_used":1756930048,"bytes_avail":288153362432,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":247134,"degraded_ratio":0.13782806089004346,"misplaced_objects":3210,"misplaced_total":247134,"misplaced_ratio":0.012988904804680861,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":56572,"recovering_keys_per_sec":0,"num_objects_recovered":113,"num_bytes_recovered":462848,"num_keys_recovered":0,"write_bytes_sec":1841363,"read_op_per_sec":0,"write_op_per_sec":449},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:51:40.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:51:40.839 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:41.105 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 164 20 125406 125386 2.98551 1.89844 0.0413204 0.0261605 2024-09-16T04:51:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:41.808 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-16T04:51:40.321475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506097906,"num_objects":123567,"num_object_clones":0,"num_object_copies":247134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3210,"num_objects_unfound":0,"num_objects_dirty":123567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88801,"num_write_kb":355170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3469,"num_bytes_recovered":14172179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123724,"ondisk_log_size":123724,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1715752,"kb_used_data":998768,"kb_used_omap":12,"kb_used_meta":716915,"kb_avail":281399768,"statfs":{"total":289910292480,"available":288153362432,"internally_reserved":0,"allocated":1022738432,"data_stored":1021861903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":734120961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,4,5,7,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15065088,"num_objects":3678,"num_object_clones":0,"num_object_copies":7356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3325,"num_objects_misplaced":3210,"num_objects_unfound":0,"num_objects_dirty":3678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3678,"num_write_kb":14712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":462848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.181484"},"pg_stats":[{"pgid":"2.f","version":"47'7596","reported_seq":7672,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.743615+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:37.743615+0000","last_peered":"2024-09-16T04:51:37.743615+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:37.743615+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7596,"log_dups_size":0,"ondisk_log_size":7596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31113216,"num_objects":7596,"num_object_clones":0,"num_object_copies":15192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7596,"num_write_kb":30384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'7693","reported_seq":7809,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:36.943531+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:36.943531+0000","last_peered":"2024-09-16T04:51:36.943531+0000","last_clean":"2024-09-16T04:51:36.943531+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:36.943531+0000","last_undegraded":"2024-09-16T04:51:36.943531+0000","last_fullsized":"2024-09-16T04:51:36.943531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7693,"log_dups_size":0,"ondisk_log_size":7693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31485952,"num_objects":7687,"num_object_clones":0,"num_object_copies":15374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4662,"num_write_kb":18648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'7637","reported_seq":7775,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.014782+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:37.014782+0000","last_peered":"2024-09-16T04:51:37.014782+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:37.014782+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7637,"log_dups_size":0,"ondisk_log_size":7637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31264768,"num_objects":7633,"num_object_clones":0,"num_object_copies":15266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4376,"num_write_kb":17504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4314}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'7780","reported_seq":7850,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:36.986184+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:36.986184+0000","last_peered":"2024-09-16T04:51:36.986184+0000","last_clean":"2024-09-16T04:51:36.986184+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:36.986184+0000","last_undegraded":"2024-09-16T04:51:36.986184+0000","last_fullsized":"2024-09-16T04:51:36.986184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7780,"log_dups_size":0,"ondisk_log_size":7780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31866880,"num_objects":7780,"num_object_clones":0,"num_object_copies":15560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7780,"num_write_kb":31120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'7792","reported_seq":8010,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:37.623882+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:37.623882+0000","last_peered":"2024-09-16T04:51:37.623882+0000","last_clean":"2024-09-16T04:51:37.623882+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:37.623882+0000","last_undegraded":"2024-09-16T04:51:37.623882+0000","last_fullsized":"2024-09-16T04:51:37.623882+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7792,"log_dups_size":0,"ondisk_log_size":7792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31793152,"num_objects":7762,"num_object_clones":0,"num_object_copies":15524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4824,"num_write_kb":19296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'7815","reported_seq":7948,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.638148+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:37.638148+0000","last_peered":"2024-09-16T04:51:37.638148+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:37.638148+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7815,"log_dups_size":0,"ondisk_log_size":7815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31989760,"num_objects":7810,"num_object_clones":0,"num_object_copies":15620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4808,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4354},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'7686","reported_seq":7833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.626083+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:37.626083+0000","last_peered":"2024-09-16T04:51:37.626083+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:37.626083+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7686,"log_dups_size":0,"ondisk_log_size":7686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31457280,"num_objects":7680,"num_object_clones":0,"num_object_copies":15360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4341,"num_write_kb":17364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4281},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'7713","reported_seq":7929,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:37.694365+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:37.694365+0000","last_peered":"2024-09-16T04:51:37.694365+0000","last_clean":"2024-09-16T04:51:37.694365+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:37.694365+0000","last_undegraded":"2024-09-16T04:51:37.694365+0000","last_fullsized":"2024-09-16T04:51:37.694365+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7713,"log_dups_size":0,"ondisk_log_size":7713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31559680,"num_objects":7705,"num_object_clones":0,"num_object_copies":15410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4783,"num_write_kb":19132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'7745","reported_seq":7973,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:36.991629+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:36.991629+0000","last_peered":"2024-09-16T04:51:36.991629+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:36.991629+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7745,"log_dups_size":0,"ondisk_log_size":7745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31588352,"num_objects":7712,"num_object_clones":0,"num_object_copies":15424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4496,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":4391}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'7793","reported_seq":7919,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.707458+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:37.707458+0000","last_peered":"2024-09-16T04:51:37.707458+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:37.707458+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7793,"log_dups_size":0,"ondisk_log_size":7793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31887360,"num_objects":7785,"num_object_clones":0,"num_object_copies":15570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4448,"num_write_kb":17792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4320},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'7637","reported_seq":7879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.017380+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:37.017380+0000","last_peered":"2024-09-16T04:51:37.017380+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:37.017380+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7637,"log_dups_size":0,"ondisk_log_size":7637,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31155410,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7609,"num_write_kb":30427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4312}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":73,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149422+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149422+0000","last_peered":"2024-09-16T04:51:33.149422+0000","last_clean":"2024-09-16T04:51:33.149422+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149422+0000","last_undegraded":"2024-09-16T04:51:33.149422+0000","last_fullsized":"2024-09-16T04:51:33.149422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'7694","reported_seq":14550,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:37.005085+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:51:37.005085+0000","last_peered":"2024-09-16T04:51:37.005085+0000","last_clean":"2024-09-16T04:51:37.005085+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:51:37.005085+0000","last_undegraded":"2024-09-16T04:51:37.005085+0000","last_fullsized":"2024-09-16T04:51:37.005085+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7694,"log_dups_size":0,"ondisk_log_size":7694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31490048,"num_objects":7688,"num_object_clones":0,"num_object_copies":15376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4417,"num_write_kb":17668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'7611","reported_seq":7708,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:36.964379+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:36.964379+0000","last_peered":"2024-09-16T04:51:36.964379+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:36.964379+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7611,"log_dups_size":0,"ondisk_log_size":7611,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31167081,"num_objects":7610,"num_object_clones":0,"num_object_copies":15220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7610,"num_write_kb":30437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4203}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149652+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149652+0000","last_peered":"2024-09-16T04:51:33.149652+0000","last_clean":"2024-09-16T04:51:33.149652+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149652+0000","last_undegraded":"2024-09-16T04:51:33.149652+0000","last_fullsized":"2024-09-16T04:51:33.149652+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'7878","reported_seq":7980,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.019635+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:37.019635+0000","last_peered":"2024-09-16T04:51:37.019635+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:37.019635+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":7878,"log_dups_size":0,"ondisk_log_size":7878,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32237371,"num_objects":7873,"num_object_clones":0,"num_object_copies":15746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7875,"num_write_kb":31484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4359}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'7786","reported_seq":8144,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.712314+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:51:37.712314+0000","last_peered":"2024-09-16T04:51:37.712314+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:37.712314+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7786,"log_dups_size":0,"ondisk_log_size":7786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31875072,"num_objects":7782,"num_object_clones":0,"num_object_copies":15564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3210,"num_objects_unfound":0,"num_objects_dirty":7782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4540,"num_write_kb":18160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":462848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4570},{"shards":"0,2","objects":3210}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150640+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150640+0000","last_peered":"2024-09-16T04:51:33.150640+0000","last_clean":"2024-09-16T04:51:33.150640+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150640+0000","last_undegraded":"2024-09-16T04:51:33.150640+0000","last_fullsized":"2024-09-16T04:51:33.150640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155436+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155436+0000","last_peered":"2024-09-16T04:51:33.155436+0000","last_clean":"2024-09-16T04:51:33.155436+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155436+0000","last_undegraded":"2024-09-16T04:51:33.155436+0000","last_fullsized":"2024-09-16T04:51:33.155436+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'7865","reported_seq":8005,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:37.630008+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:37.630008+0000","last_peered":"2024-09-16T04:51:37.630008+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:37.630008+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":7865,"log_dups_size":0,"ondisk_log_size":7865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32166505,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4634,"num_write_kb":18536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4330},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":506097887,"num_objects":123565,"num_object_clones":0,"num_object_copies":247130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3210,"num_objects_unfound":0,"num_objects_dirty":123565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88799,"num_write_kb":355168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3468,"num_bytes_recovered":14172160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1023385600,"data_stored":1023305583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123721,"ondisk_log_size":123721,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664856,"kb_used_data":390212,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93706984,"statfs":{"total":96636764160,"available":95955951616,"internally_reserved":0,"allocated":399577088,"data_stored":399308105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,3,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593753,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394644,"kb_used_data":218560,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93977196,"statfs":{"total":96636764160,"available":96232648704,"internally_reserved":0,"allocated":223805440,"data_stored":223466877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":180285102},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656252,"kb_used_data":389996,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93715588,"statfs":{"total":96636764160,"available":95964762112,"internally_reserved":0,"allocated":399355904,"data_stored":399086921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":400175104,"data_stored":400171625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":223719424,"data_stored":223646365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":399491072,"data_stored":399487593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:42.101 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 165 20 125906 125886 2.97925 1.95312 0.0159389 0.0262153 2024-09-16T04:51:43.101 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 166 20 126396 126376 2.97283 1.91406 0.0381599 0.0262717 2024-09-16T04:51:44.101 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 167 20 126902 126882 2.96687 1.97656 0.00625144 0.0263245 2024-09-16T04:51:44.810 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:45.102 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 168 20 127407 127387 2.96095 1.97266 0.0125248 0.0263754 2024-09-16T04:51:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:45.710 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:45.787 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-16T04:51:44.323305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":516223218,"num_objects":126039,"num_object_clones":0,"num_object_copies":252078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":126039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91273,"num_write_kb":365058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":14487571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126196,"ondisk_log_size":126196,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1735996,"kb_used_data":1019020,"kb_used_omap":12,"kb_used_meta":716915,"kb_avail":281379524,"statfs":{"total":289910292480,"available":288132632576,"internally_reserved":0,"allocated":1043476480,"data_stored":1042599951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":734120961},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,6,6,6,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20111360,"num_objects":4910,"num_object_clones":0,"num_object_copies":9820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-167,"num_objects_unfound":0,"num_objects_dirty":4910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4910,"num_write_kb":19640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":684032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.176382"},"pg_stats":[{"pgid":"2.f","version":"47'7736","reported_seq":7815,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.694001+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:42.694001+0000","last_peered":"2024-09-16T04:51:42.694001+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:42.694001+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7736,"log_dups_size":0,"ondisk_log_size":7736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31686656,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7736,"num_write_kb":30944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4395},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'7867","reported_seq":7983,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:42.051730+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:42.051730+0000","last_peered":"2024-09-16T04:51:42.051730+0000","last_clean":"2024-09-16T04:51:42.051730+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:42.051730+0000","last_undegraded":"2024-09-16T04:51:42.051730+0000","last_fullsized":"2024-09-16T04:51:42.051730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7867,"log_dups_size":0,"ondisk_log_size":7867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32198656,"num_objects":7861,"num_object_clones":0,"num_object_copies":15722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4836,"num_write_kb":19344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'7798","reported_seq":7934,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.053841+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:42.053841+0000","last_peered":"2024-09-16T04:51:42.053841+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:42.053841+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7798,"log_dups_size":0,"ondisk_log_size":7798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31924224,"num_objects":7794,"num_object_clones":0,"num_object_copies":15588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4537,"num_write_kb":18148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4475}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'7928","reported_seq":7998,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:42.025366+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:42.025366+0000","last_peered":"2024-09-16T04:51:42.025366+0000","last_clean":"2024-09-16T04:51:42.025366+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:42.025366+0000","last_undegraded":"2024-09-16T04:51:42.025366+0000","last_fullsized":"2024-09-16T04:51:42.025366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7928,"log_dups_size":0,"ondisk_log_size":7928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32473088,"num_objects":7928,"num_object_clones":0,"num_object_copies":15856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7928,"num_write_kb":31712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'7939","reported_seq":8157,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:42.706107+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:42.706107+0000","last_peered":"2024-09-16T04:51:42.706107+0000","last_clean":"2024-09-16T04:51:42.706107+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:42.706107+0000","last_undegraded":"2024-09-16T04:51:42.706107+0000","last_fullsized":"2024-09-16T04:51:42.706107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7939,"log_dups_size":0,"ondisk_log_size":7939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32395264,"num_objects":7909,"num_object_clones":0,"num_object_copies":15818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4971,"num_write_kb":19884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'7969","reported_seq":8102,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.696416+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:42.696416+0000","last_peered":"2024-09-16T04:51:42.696416+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:42.696416+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7969,"log_dups_size":0,"ondisk_log_size":7969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32620544,"num_objects":7964,"num_object_clones":0,"num_object_copies":15928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4962,"num_write_kb":19848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4508},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'7827","reported_seq":7975,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.734930+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:42.734930+0000","last_peered":"2024-09-16T04:51:42.734930+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:42.734930+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7827,"log_dups_size":0,"ondisk_log_size":7827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32034816,"num_objects":7821,"num_object_clones":0,"num_object_copies":15642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4482,"num_write_kb":17928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4422},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'7865","reported_seq":8081,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:42.666779+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:42.666779+0000","last_peered":"2024-09-16T04:51:42.666779+0000","last_clean":"2024-09-16T04:51:42.666779+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:42.666779+0000","last_undegraded":"2024-09-16T04:51:42.666779+0000","last_fullsized":"2024-09-16T04:51:42.666779+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7865,"log_dups_size":0,"ondisk_log_size":7865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32182272,"num_objects":7857,"num_object_clones":0,"num_object_copies":15714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4935,"num_write_kb":19740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'7904","reported_seq":8132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:41.962041+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:41.962041+0000","last_peered":"2024-09-16T04:51:41.962041+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:41.962041+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7904,"log_dups_size":0,"ondisk_log_size":7904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32239616,"num_objects":7871,"num_object_clones":0,"num_object_copies":15742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":4550}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'7958","reported_seq":8087,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.720433+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:42.720433+0000","last_peered":"2024-09-16T04:51:42.720433+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:42.720433+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7958,"log_dups_size":0,"ondisk_log_size":7958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32563200,"num_objects":7950,"num_object_clones":0,"num_object_copies":15900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4613,"num_write_kb":18452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4485},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'7791","reported_seq":8033,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:41.964042+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:41.964042+0000","last_peered":"2024-09-16T04:51:41.964042+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:41.964042+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7791,"log_dups_size":0,"ondisk_log_size":7791,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31786194,"num_objects":7762,"num_object_clones":0,"num_object_copies":15524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7763,"num_write_kb":31043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4466}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'7833","reported_seq":14687,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:41.962903+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:51:41.962903+0000","last_peered":"2024-09-16T04:51:41.962903+0000","last_clean":"2024-09-16T04:51:41.962903+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:51:41.962903+0000","last_undegraded":"2024-09-16T04:51:41.962903+0000","last_fullsized":"2024-09-16T04:51:41.962903+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7833,"log_dups_size":0,"ondisk_log_size":7833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32059392,"num_objects":7827,"num_object_clones":0,"num_object_copies":15654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4556,"num_write_kb":18224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'7796","reported_seq":7893,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.035720+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:42.035720+0000","last_peered":"2024-09-16T04:51:42.035720+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:42.035720+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7796,"log_dups_size":0,"ondisk_log_size":7796,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31924841,"num_objects":7795,"num_object_clones":0,"num_object_copies":15590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7795,"num_write_kb":31177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4388}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8023","reported_seq":8125,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:41.981552+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:41.981552+0000","last_peered":"2024-09-16T04:51:41.981552+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:41.981552+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8023,"log_dups_size":0,"ondisk_log_size":8023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32831291,"num_objects":8018,"num_object_clones":0,"num_object_copies":16036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8020,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4504}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'7949","reported_seq":8461,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.683115+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:51:42.683115+0000","last_peered":"2024-09-16T04:51:42.683115+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:42.683115+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7949,"log_dups_size":0,"ondisk_log_size":7949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32542720,"num_objects":7945,"num_object_clones":0,"num_object_copies":15890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":7945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4703,"num_write_kb":18812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":778240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4810},{"shards":"0,2","objects":3133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150640+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150640+0000","last_peered":"2024-09-16T04:51:33.150640+0000","last_clean":"2024-09-16T04:51:33.150640+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150640+0000","last_undegraded":"2024-09-16T04:51:33.150640+0000","last_fullsized":"2024-09-16T04:51:33.150640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155436+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155436+0000","last_peered":"2024-09-16T04:51:33.155436+0000","last_clean":"2024-09-16T04:51:33.155436+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155436+0000","last_undegraded":"2024-09-16T04:51:33.155436+0000","last_fullsized":"2024-09-16T04:51:33.155436+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8010","reported_seq":8150,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.722030+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:42.722030+0000","last_peered":"2024-09-16T04:51:42.722030+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:42.722030+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8010,"log_dups_size":0,"ondisk_log_size":8010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32760425,"num_objects":7999,"num_object_clones":0,"num_object_copies":15998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4779,"num_write_kb":19116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4475},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":516223199,"num_objects":126037,"num_object_clones":0,"num_object_copies":252074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":126037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91271,"num_write_kb":365056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14487552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1044172800,"data_stored":1044092783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126193,"ondisk_log_size":126193,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658560,"kb_used_data":396400,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93713280,"statfs":{"total":96636764160,"available":95962398720,"internally_reserved":0,"allocated":405913600,"data_stored":405644617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593754,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406732,"kb_used_data":226492,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93965108,"statfs":{"total":96636764160,"available":96220270592,"internally_reserved":0,"allocated":231927808,"data_stored":231589245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184544942},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670704,"kb_used_data":396128,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93701136,"statfs":{"total":96636764160,"available":95949963264,"internally_reserved":0,"allocated":405635072,"data_stored":405366089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":406413312,"data_stored":406409833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231899136,"data_stored":231826077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405860352,"data_stored":405856873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:45.789 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:46.102 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 169 20 127891 127871 2.95462 1.89062 0.0221114 0.0264339 2024-09-16T04:51:46.555 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:46.556 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:46.619 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-16T04:51:46.333333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":516223218,"num_objects":126039,"num_object_clones":0,"num_object_copies":252078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":126039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91273,"num_write_kb":365058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":14487571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126196,"ondisk_log_size":126196,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1770128,"kb_used_data":1026784,"kb_used_omap":12,"kb_used_meta":743283,"kb_avail":281345392,"statfs":{"total":289910292480,"available":288097681408,"internally_reserved":0,"allocated":1051426816,"data_stored":1050550287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":761121793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,5,9,3,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20111360,"num_objects":4910,"num_object_clones":0,"num_object_copies":9820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-167,"num_objects_unfound":0,"num_objects_dirty":4910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4910,"num_write_kb":19640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":684032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013485"},"pg_stats":[{"pgid":"2.f","version":"47'7736","reported_seq":7815,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.694001+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:42.694001+0000","last_peered":"2024-09-16T04:51:42.694001+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:42.694001+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7736,"log_dups_size":0,"ondisk_log_size":7736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31686656,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7736,"num_write_kb":30944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4395},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'7867","reported_seq":7983,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:42.051730+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:42.051730+0000","last_peered":"2024-09-16T04:51:42.051730+0000","last_clean":"2024-09-16T04:51:42.051730+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:42.051730+0000","last_undegraded":"2024-09-16T04:51:42.051730+0000","last_fullsized":"2024-09-16T04:51:42.051730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7867,"log_dups_size":0,"ondisk_log_size":7867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32198656,"num_objects":7861,"num_object_clones":0,"num_object_copies":15722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4836,"num_write_kb":19344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'7798","reported_seq":7934,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.053841+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:42.053841+0000","last_peered":"2024-09-16T04:51:42.053841+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:42.053841+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7798,"log_dups_size":0,"ondisk_log_size":7798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31924224,"num_objects":7794,"num_object_clones":0,"num_object_copies":15588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4537,"num_write_kb":18148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4475}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'7928","reported_seq":7998,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:42.025366+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:42.025366+0000","last_peered":"2024-09-16T04:51:42.025366+0000","last_clean":"2024-09-16T04:51:42.025366+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:42.025366+0000","last_undegraded":"2024-09-16T04:51:42.025366+0000","last_fullsized":"2024-09-16T04:51:42.025366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7928,"log_dups_size":0,"ondisk_log_size":7928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32473088,"num_objects":7928,"num_object_clones":0,"num_object_copies":15856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7928,"num_write_kb":31712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'7939","reported_seq":8157,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:42.706107+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:42.706107+0000","last_peered":"2024-09-16T04:51:42.706107+0000","last_clean":"2024-09-16T04:51:42.706107+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:42.706107+0000","last_undegraded":"2024-09-16T04:51:42.706107+0000","last_fullsized":"2024-09-16T04:51:42.706107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7939,"log_dups_size":0,"ondisk_log_size":7939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32395264,"num_objects":7909,"num_object_clones":0,"num_object_copies":15818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4971,"num_write_kb":19884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'7969","reported_seq":8102,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.696416+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:42.696416+0000","last_peered":"2024-09-16T04:51:42.696416+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:42.696416+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7969,"log_dups_size":0,"ondisk_log_size":7969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32620544,"num_objects":7964,"num_object_clones":0,"num_object_copies":15928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4962,"num_write_kb":19848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4508},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'7827","reported_seq":7975,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.734930+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:42.734930+0000","last_peered":"2024-09-16T04:51:42.734930+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:42.734930+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7827,"log_dups_size":0,"ondisk_log_size":7827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32034816,"num_objects":7821,"num_object_clones":0,"num_object_copies":15642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4482,"num_write_kb":17928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4422},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'7865","reported_seq":8081,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:42.666779+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:42.666779+0000","last_peered":"2024-09-16T04:51:42.666779+0000","last_clean":"2024-09-16T04:51:42.666779+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:42.666779+0000","last_undegraded":"2024-09-16T04:51:42.666779+0000","last_fullsized":"2024-09-16T04:51:42.666779+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7865,"log_dups_size":0,"ondisk_log_size":7865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32182272,"num_objects":7857,"num_object_clones":0,"num_object_copies":15714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4935,"num_write_kb":19740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'7904","reported_seq":8132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:41.962041+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:41.962041+0000","last_peered":"2024-09-16T04:51:41.962041+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:41.962041+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7904,"log_dups_size":0,"ondisk_log_size":7904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32239616,"num_objects":7871,"num_object_clones":0,"num_object_copies":15742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":4550}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'7958","reported_seq":8087,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.720433+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:42.720433+0000","last_peered":"2024-09-16T04:51:42.720433+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:42.720433+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7958,"log_dups_size":0,"ondisk_log_size":7958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32563200,"num_objects":7950,"num_object_clones":0,"num_object_copies":15900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4613,"num_write_kb":18452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4485},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'7791","reported_seq":8033,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:41.964042+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:41.964042+0000","last_peered":"2024-09-16T04:51:41.964042+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:41.964042+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7791,"log_dups_size":0,"ondisk_log_size":7791,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31786194,"num_objects":7762,"num_object_clones":0,"num_object_copies":15524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7763,"num_write_kb":31043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4466}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'7833","reported_seq":14687,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:41.962903+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:51:41.962903+0000","last_peered":"2024-09-16T04:51:41.962903+0000","last_clean":"2024-09-16T04:51:41.962903+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:51:41.962903+0000","last_undegraded":"2024-09-16T04:51:41.962903+0000","last_fullsized":"2024-09-16T04:51:41.962903+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7833,"log_dups_size":0,"ondisk_log_size":7833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32059392,"num_objects":7827,"num_object_clones":0,"num_object_copies":15654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4556,"num_write_kb":18224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'7796","reported_seq":7893,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.035720+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:42.035720+0000","last_peered":"2024-09-16T04:51:42.035720+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:42.035720+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7796,"log_dups_size":0,"ondisk_log_size":7796,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31924841,"num_objects":7795,"num_object_clones":0,"num_object_copies":15590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7795,"num_write_kb":31177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4388}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8023","reported_seq":8125,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:41.981552+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:41.981552+0000","last_peered":"2024-09-16T04:51:41.981552+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:41.981552+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8023,"log_dups_size":0,"ondisk_log_size":8023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32831291,"num_objects":8018,"num_object_clones":0,"num_object_copies":16036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8020,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4504}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'7949","reported_seq":8461,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.683115+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:51:42.683115+0000","last_peered":"2024-09-16T04:51:42.683115+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:42.683115+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7949,"log_dups_size":0,"ondisk_log_size":7949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32542720,"num_objects":7945,"num_object_clones":0,"num_object_copies":15890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":7945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4703,"num_write_kb":18812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":778240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4810},{"shards":"0,2","objects":3133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8010","reported_seq":8150,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.722030+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:42.722030+0000","last_peered":"2024-09-16T04:51:42.722030+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:42.722030+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8010,"log_dups_size":0,"ondisk_log_size":8010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32760425,"num_objects":7999,"num_object_clones":0,"num_object_copies":15998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4779,"num_write_kb":19116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4475},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":516223199,"num_objects":126037,"num_object_clones":0,"num_object_copies":252074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":126037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91271,"num_write_kb":365056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14487552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1052319744,"data_stored":1052239727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126193,"ondisk_log_size":126193,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658560,"kb_used_data":396400,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93713280,"statfs":{"total":96636764160,"available":95962398720,"internally_reserved":0,"allocated":405913600,"data_stored":405644617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593755,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":440864,"kb_used_data":234256,"kb_used_omap":4,"kb_used_meta":206587,"kb_avail":93930976,"statfs":{"total":96636764160,"available":96185319424,"internally_reserved":0,"allocated":239878144,"data_stored":239539581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":211545774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670704,"kb_used_data":396128,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93701136,"statfs":{"total":96636764160,"available":95949963264,"internally_reserved":0,"allocated":405635072,"data_stored":405366089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":406413312,"data_stored":406409833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":240046080,"data_stored":239973021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405860352,"data_stored":405856873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:46.620 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:51:46.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'7736", 'reported_seq': 7815, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:42.694001+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:51:42.694001+0000', 'last_peered': '2024-09-16T04:51:42.694001+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:51:42.694001+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 7736, 'log_dups_size': 0, 'ondisk_log_size': 7736, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31686656, 'num_objects': 7736, 'num_object_clones': 0, 'num_object_copies': 15472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7736, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7736, 'num_write_kb': 30944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4395}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:46.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:51:46.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'7798", 'reported_seq': 7934, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:42.053841+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:51:42.053841+0000', 'last_peered': '2024-09-16T04:51:42.053841+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:51:42.053841+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 7798, 'log_dups_size': 0, 'ondisk_log_size': 7798, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31924224, 'num_objects': 7794, 'num_object_clones': 0, 'num_object_copies': 15588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7794, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4537, 'num_write_kb': 18148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 4475}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:46.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:51:46.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'7969", 'reported_seq': 8102, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:42.696416+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:51:42.696416+0000', 'last_peered': '2024-09-16T04:51:42.696416+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:51:42.696416+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 7969, 'log_dups_size': 0, 'ondisk_log_size': 7969, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32620544, 'num_objects': 7964, 'num_object_clones': 0, 'num_object_copies': 15928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7964, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4962, 'num_write_kb': 19848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4508}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:46.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:51:46.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'7827", 'reported_seq': 7975, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:42.734930+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:51:42.734930+0000', 'last_peered': '2024-09-16T04:51:42.734930+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:51:42.734930+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 7827, 'log_dups_size': 0, 'ondisk_log_size': 7827, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32034816, 'num_objects': 7821, 'num_object_clones': 0, 'num_object_copies': 15642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7821, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4482, 'num_write_kb': 17928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4422}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:46.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:51:46.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'7904", 'reported_seq': 8132, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:41.962041+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:51:41.962041+0000', 'last_peered': '2024-09-16T04:51:41.962041+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:51:41.962041+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 7904, 'log_dups_size': 0, 'ondisk_log_size': 7904, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32239616, 'num_objects': 7871, 'num_object_clones': 0, 'num_object_copies': 15742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7871, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4655, 'num_write_kb': 18620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 4550}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:46.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:51:46.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'7958", 'reported_seq': 8087, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:42.720433+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:51:42.720433+0000', 'last_peered': '2024-09-16T04:51:42.720433+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:51:42.720433+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 7958, 'log_dups_size': 0, 'ondisk_log_size': 7958, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32563200, 'num_objects': 7950, 'num_object_clones': 0, 'num_object_copies': 15900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7950, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4613, 'num_write_kb': 18452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4485}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:46.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:51:46.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'7791", 'reported_seq': 8033, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:41.964042+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:51:41.964042+0000', 'last_peered': '2024-09-16T04:51:41.964042+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:51:41.964042+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 7791, 'log_dups_size': 0, 'ondisk_log_size': 7791, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31786194, 'num_objects': 7762, 'num_object_clones': 0, 'num_object_copies': 15524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7763, 'num_write_kb': 31043, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 4466}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:46.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:51:46.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'7796", 'reported_seq': 7893, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:42.035720+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:51:42.035720+0000', 'last_peered': '2024-09-16T04:51:42.035720+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:51:42.035720+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 7796, 'log_dups_size': 0, 'ondisk_log_size': 7796, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31924841, 'num_objects': 7795, 'num_object_clones': 0, 'num_object_copies': 15590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7795, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7795, 'num_write_kb': 31177, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 4388}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:46.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:51:46.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'8023", 'reported_seq': 8125, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:41.981552+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:51:41.981552+0000', 'last_peered': '2024-09-16T04:51:41.981552+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:51:41.981552+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 8023, 'log_dups_size': 0, 'ondisk_log_size': 8023, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32831291, 'num_objects': 8018, 'num_object_clones': 0, 'num_object_copies': 16036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8018, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8020, 'num_write_kb': 32064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 4504}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:46.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:51:46.622 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'7949", 'reported_seq': 8461, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:42.683115+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:51:42.683115+0000', 'last_peered': '2024-09-16T04:51:42.683115+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:51:42.683115+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 7949, 'log_dups_size': 0, 'ondisk_log_size': 7949, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32542720, 'num_objects': 7945, 'num_object_clones': 0, 'num_object_copies': 15890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3133, 'num_objects_unfound': 0, 'num_objects_dirty': 7945, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4703, 'num_write_kb': 18812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 190, 'num_bytes_recovered': 778240, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4810}, {'shards': '0,2', 'objects': 3133}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:46.623 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:51:46.623 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'8010", 'reported_seq': 8150, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:42.722030+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:51:42.722030+0000', 'last_peered': '2024-09-16T04:51:42.722030+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:51:42.722030+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 8010, 'log_dups_size': 0, 'ondisk_log_size': 8010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32760425, 'num_objects': 7999, 'num_object_clones': 0, 'num_object_copies': 15998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7999, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4779, 'num_write_kb': 19116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4475}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:46.623 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:51:47.105 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 170 20 128362 128342 2.94801 1.83984 0.0387324 0.0264927 2024-09-16T04:51:47.498 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:47.498 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/252078 objects degraded (13.512%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":255,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":126039,"data_bytes":516223218,"bytes_used":1812611072,"bytes_avail":288097681408,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":252078,"degraded_ratio":0.13512484231071334,"misplaced_objects":3133,"misplaced_total":252078,"misplaced_ratio":0.01242869270622585,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":56938,"recovering_keys_per_sec":0,"num_objects_recovered":167,"num_bytes_recovered":684032,"num_keys_recovered":0,"write_bytes_sec":1674065,"read_op_per_sec":0,"write_op_per_sec":408},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:51:47.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:51:47.583 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:48.105 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 171 20 128831 128811 2.94149 1.83203 0.0299847 0.0265502 2024-09-16T04:51:48.338 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:48.340 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:48.380 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-16T04:51:46.333333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":516223218,"num_objects":126039,"num_object_clones":0,"num_object_copies":252078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":126039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91273,"num_write_kb":365058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":14487571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126196,"ondisk_log_size":126196,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1770128,"kb_used_data":1026784,"kb_used_omap":12,"kb_used_meta":743283,"kb_avail":281345392,"statfs":{"total":289910292480,"available":288097681408,"internally_reserved":0,"allocated":1051426816,"data_stored":1050550287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":761121793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,5,9,3,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20111360,"num_objects":4910,"num_object_clones":0,"num_object_copies":9820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-167,"num_objects_unfound":0,"num_objects_dirty":4910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4910,"num_write_kb":19640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":684032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013485"},"pg_stats":[{"pgid":"2.f","version":"47'7736","reported_seq":7815,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.694001+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:42.694001+0000","last_peered":"2024-09-16T04:51:42.694001+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:42.694001+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7736,"log_dups_size":0,"ondisk_log_size":7736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31686656,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7736,"num_write_kb":30944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4395},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'7867","reported_seq":7983,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:42.051730+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:42.051730+0000","last_peered":"2024-09-16T04:51:42.051730+0000","last_clean":"2024-09-16T04:51:42.051730+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:42.051730+0000","last_undegraded":"2024-09-16T04:51:42.051730+0000","last_fullsized":"2024-09-16T04:51:42.051730+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":7867,"log_dups_size":0,"ondisk_log_size":7867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32198656,"num_objects":7861,"num_object_clones":0,"num_object_copies":15722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4836,"num_write_kb":19344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'7798","reported_seq":7934,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.053841+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:42.053841+0000","last_peered":"2024-09-16T04:51:42.053841+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:42.053841+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7798,"log_dups_size":0,"ondisk_log_size":7798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31924224,"num_objects":7794,"num_object_clones":0,"num_object_copies":15588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4537,"num_write_kb":18148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4475}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'7928","reported_seq":7998,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:42.025366+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:42.025366+0000","last_peered":"2024-09-16T04:51:42.025366+0000","last_clean":"2024-09-16T04:51:42.025366+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:42.025366+0000","last_undegraded":"2024-09-16T04:51:42.025366+0000","last_fullsized":"2024-09-16T04:51:42.025366+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7928,"log_dups_size":0,"ondisk_log_size":7928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32473088,"num_objects":7928,"num_object_clones":0,"num_object_copies":15856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7928,"num_write_kb":31712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'7939","reported_seq":8157,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:42.706107+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:42.706107+0000","last_peered":"2024-09-16T04:51:42.706107+0000","last_clean":"2024-09-16T04:51:42.706107+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:42.706107+0000","last_undegraded":"2024-09-16T04:51:42.706107+0000","last_fullsized":"2024-09-16T04:51:42.706107+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":7939,"log_dups_size":0,"ondisk_log_size":7939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32395264,"num_objects":7909,"num_object_clones":0,"num_object_copies":15818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4971,"num_write_kb":19884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'7969","reported_seq":8102,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.696416+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:42.696416+0000","last_peered":"2024-09-16T04:51:42.696416+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:42.696416+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":7969,"log_dups_size":0,"ondisk_log_size":7969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32620544,"num_objects":7964,"num_object_clones":0,"num_object_copies":15928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4962,"num_write_kb":19848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4508},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'7827","reported_seq":7975,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.734930+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:42.734930+0000","last_peered":"2024-09-16T04:51:42.734930+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:42.734930+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":7827,"log_dups_size":0,"ondisk_log_size":7827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32034816,"num_objects":7821,"num_object_clones":0,"num_object_copies":15642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4482,"num_write_kb":17928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4422},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'7865","reported_seq":8081,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:42.666779+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:42.666779+0000","last_peered":"2024-09-16T04:51:42.666779+0000","last_clean":"2024-09-16T04:51:42.666779+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:42.666779+0000","last_undegraded":"2024-09-16T04:51:42.666779+0000","last_fullsized":"2024-09-16T04:51:42.666779+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":7865,"log_dups_size":0,"ondisk_log_size":7865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32182272,"num_objects":7857,"num_object_clones":0,"num_object_copies":15714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4935,"num_write_kb":19740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'7904","reported_seq":8132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:41.962041+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:41.962041+0000","last_peered":"2024-09-16T04:51:41.962041+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:41.962041+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":7904,"log_dups_size":0,"ondisk_log_size":7904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32239616,"num_objects":7871,"num_object_clones":0,"num_object_copies":15742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":4550}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'7958","reported_seq":8087,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.720433+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:42.720433+0000","last_peered":"2024-09-16T04:51:42.720433+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:42.720433+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":7958,"log_dups_size":0,"ondisk_log_size":7958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32563200,"num_objects":7950,"num_object_clones":0,"num_object_copies":15900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4613,"num_write_kb":18452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4485},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'7791","reported_seq":8033,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:41.964042+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:41.964042+0000","last_peered":"2024-09-16T04:51:41.964042+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:41.964042+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7791,"log_dups_size":0,"ondisk_log_size":7791,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31786194,"num_objects":7762,"num_object_clones":0,"num_object_copies":15524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7763,"num_write_kb":31043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4466}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'7833","reported_seq":14687,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:41.962903+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:51:41.962903+0000","last_peered":"2024-09-16T04:51:41.962903+0000","last_clean":"2024-09-16T04:51:41.962903+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:51:41.962903+0000","last_undegraded":"2024-09-16T04:51:41.962903+0000","last_fullsized":"2024-09-16T04:51:41.962903+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7833,"log_dups_size":0,"ondisk_log_size":7833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32059392,"num_objects":7827,"num_object_clones":0,"num_object_copies":15654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4556,"num_write_kb":18224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'7796","reported_seq":7893,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.035720+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:42.035720+0000","last_peered":"2024-09-16T04:51:42.035720+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:42.035720+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7796,"log_dups_size":0,"ondisk_log_size":7796,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31924841,"num_objects":7795,"num_object_clones":0,"num_object_copies":15590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7795,"num_write_kb":31177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4388}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8023","reported_seq":8125,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:41.981552+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:41.981552+0000","last_peered":"2024-09-16T04:51:41.981552+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:41.981552+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8023,"log_dups_size":0,"ondisk_log_size":8023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32831291,"num_objects":8018,"num_object_clones":0,"num_object_copies":16036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8020,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4504}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'7949","reported_seq":8461,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.683115+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:51:42.683115+0000","last_peered":"2024-09-16T04:51:42.683115+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:42.683115+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":7949,"log_dups_size":0,"ondisk_log_size":7949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32542720,"num_objects":7945,"num_object_clones":0,"num_object_copies":15890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":7945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4703,"num_write_kb":18812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":778240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4810},{"shards":"0,2","objects":3133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8010","reported_seq":8150,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:42.722030+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:42.722030+0000","last_peered":"2024-09-16T04:51:42.722030+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:42.722030+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8010,"log_dups_size":0,"ondisk_log_size":8010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32760425,"num_objects":7999,"num_object_clones":0,"num_object_copies":15998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4779,"num_write_kb":19116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4475},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":516223199,"num_objects":126037,"num_object_clones":0,"num_object_copies":252074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3133,"num_objects_unfound":0,"num_objects_dirty":126037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91271,"num_write_kb":365056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14487552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1052319744,"data_stored":1052239727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126193,"ondisk_log_size":126193,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658560,"kb_used_data":396400,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93713280,"statfs":{"total":96636764160,"available":95962398720,"internally_reserved":0,"allocated":405913600,"data_stored":405644617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593755,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":440864,"kb_used_data":234256,"kb_used_omap":4,"kb_used_meta":206587,"kb_avail":93930976,"statfs":{"total":96636764160,"available":96185319424,"internally_reserved":0,"allocated":239878144,"data_stored":239539581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":211545774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670704,"kb_used_data":396128,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93701136,"statfs":{"total":96636764160,"available":95949963264,"internally_reserved":0,"allocated":405635072,"data_stored":405366089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":406413312,"data_stored":406409833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":240046080,"data_stored":239973021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405860352,"data_stored":405856873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:49.105 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 172 20 129322 129302 2.93554 1.91797 0.0577493 0.0266051 2024-09-16T04:51:50.109 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 173 20 129828 129808 2.92992 1.97656 0.00875608 0.0266553 2024-09-16T04:51:51.112 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 174 20 130305 130285 2.92375 1.86328 0.0194657 0.0267132 2024-09-16T04:51:51.383 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:52.112 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 175 20 130785 130765 2.91776 1.875 0.0404025 0.0267649 2024-09-16T04:51:52.228 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:52.229 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:52.308 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-16T04:51:50.337102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":526381298,"num_objects":128519,"num_object_clones":0,"num_object_copies":257038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3050,"num_objects_unfound":0,"num_objects_dirty":128519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":93753,"num_write_kb":374978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3629,"num_bytes_recovered":14827539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128676,"ondisk_log_size":128676,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1775160,"kb_used_data":1040136,"kb_used_omap":12,"kb_used_meta":734963,"kb_avail":281340360,"statfs":{"total":289910292480,"available":288092528640,"internally_reserved":0,"allocated":1065099264,"data_stored":1064222735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":752602113},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,13,5,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20283392,"num_objects":4952,"num_object_clones":0,"num_object_copies":9904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-160,"num_objects_unfound":0,"num_objects_dirty":4952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4952,"num_write_kb":19808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.016241"},"pg_stats":[{"pgid":"2.f","version":"47'7889","reported_seq":7967,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.751817+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:47.751817+0000","last_peered":"2024-09-16T04:51:47.751817+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:47.751817+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7889,"log_dups_size":0,"ondisk_log_size":7889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313344,"num_objects":7889,"num_object_clones":0,"num_object_copies":15778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7889,"num_write_kb":31556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4548},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8006","reported_seq":8122,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:46.952298+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:46.952298+0000","last_peered":"2024-09-16T04:51:46.952298+0000","last_clean":"2024-09-16T04:51:46.952298+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:46.952298+0000","last_undegraded":"2024-09-16T04:51:46.952298+0000","last_fullsized":"2024-09-16T04:51:46.952298+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8006,"log_dups_size":0,"ondisk_log_size":8006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32768000,"num_objects":8000,"num_object_clones":0,"num_object_copies":16000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4975,"num_write_kb":19900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'7934","reported_seq":8072,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.014795+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:47.014795+0000","last_peered":"2024-09-16T04:51:47.014795+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:47.014795+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":7934,"log_dups_size":0,"ondisk_log_size":7934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32481280,"num_objects":7930,"num_object_clones":0,"num_object_copies":15860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4673,"num_write_kb":18692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4611}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'8080","reported_seq":8150,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:46.992459+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:46.992459+0000","last_peered":"2024-09-16T04:51:46.992459+0000","last_clean":"2024-09-16T04:51:46.992459+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:46.992459+0000","last_undegraded":"2024-09-16T04:51:46.992459+0000","last_fullsized":"2024-09-16T04:51:46.992459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8080,"log_dups_size":0,"ondisk_log_size":8080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33095680,"num_objects":8080,"num_object_clones":0,"num_object_copies":16160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":32320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8077","reported_seq":8296,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:47.661113+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:47.661113+0000","last_peered":"2024-09-16T04:51:47.661113+0000","last_clean":"2024-09-16T04:51:47.661113+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:47.661113+0000","last_undegraded":"2024-09-16T04:51:47.661113+0000","last_fullsized":"2024-09-16T04:51:47.661113+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960512,"num_objects":8047,"num_object_clones":0,"num_object_copies":16094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5109,"num_write_kb":20436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'8121","reported_seq":8254,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.674584+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:47.674584+0000","last_peered":"2024-09-16T04:51:47.674584+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:47.674584+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":8121,"log_dups_size":0,"ondisk_log_size":8121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33243136,"num_objects":8116,"num_object_clones":0,"num_object_copies":16232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5114,"num_write_kb":20456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4660},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8000","reported_seq":8147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.770850+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:47.770850+0000","last_peered":"2024-09-16T04:51:47.770850+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:47.770850+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8000,"log_dups_size":0,"ondisk_log_size":8000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32743424,"num_objects":7994,"num_object_clones":0,"num_object_copies":15988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4595},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8013","reported_seq":8229,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:47.713625+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:47.713625+0000","last_peered":"2024-09-16T04:51:47.713625+0000","last_clean":"2024-09-16T04:51:47.713625+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:47.713625+0000","last_undegraded":"2024-09-16T04:51:47.713625+0000","last_fullsized":"2024-09-16T04:51:47.713625+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8013,"log_dups_size":0,"ondisk_log_size":8013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32788480,"num_objects":8005,"num_object_clones":0,"num_object_copies":16010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5083,"num_write_kb":20332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8064","reported_seq":8292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.028556+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:47.028556+0000","last_peered":"2024-09-16T04:51:47.028556+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:47.028556+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8064,"log_dups_size":0,"ondisk_log_size":8064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32894976,"num_objects":8031,"num_object_clones":0,"num_object_copies":16062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4815,"num_write_kb":19260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":4710}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8116","reported_seq":8246,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.684890+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:47.684890+0000","last_peered":"2024-09-16T04:51:47.684890+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:47.684890+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8116,"log_dups_size":0,"ondisk_log_size":8116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33210368,"num_objects":8108,"num_object_clones":0,"num_object_copies":16216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4771,"num_write_kb":19084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4643},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'7945","reported_seq":8187,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.036974+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:47.036974+0000","last_peered":"2024-09-16T04:51:47.036974+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:47.036974+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":7945,"log_dups_size":0,"ondisk_log_size":7945,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32416978,"num_objects":7916,"num_object_clones":0,"num_object_copies":15832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7917,"num_write_kb":31659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4620}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'7998","reported_seq":14854,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:46.958246+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:51:46.958246+0000","last_peered":"2024-09-16T04:51:46.958246+0000","last_clean":"2024-09-16T04:51:46.958246+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:51:46.958246+0000","last_undegraded":"2024-09-16T04:51:46.958246+0000","last_fullsized":"2024-09-16T04:51:46.958246+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":7998,"log_dups_size":0,"ondisk_log_size":7998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32735232,"num_objects":7992,"num_object_clones":0,"num_object_copies":15984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4721,"num_write_kb":18884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'7964","reported_seq":8062,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.068656+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:47.068656+0000","last_peered":"2024-09-16T04:51:47.068656+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:47.068656+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7964,"log_dups_size":0,"ondisk_log_size":7964,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32612969,"num_objects":7963,"num_object_clones":0,"num_object_copies":15926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7963,"num_write_kb":31849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8190","reported_seq":8293,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.032169+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:47.032169+0000","last_peered":"2024-09-16T04:51:47.032169+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:47.032169+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8190,"log_dups_size":0,"ondisk_log_size":8190,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33515323,"num_objects":8185,"num_object_clones":0,"num_object_copies":16370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8187,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4671}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8114","reported_seq":8792,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.764236+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:51:47.764236+0000","last_peered":"2024-09-16T04:51:47.764236+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:47.764236+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8114,"log_dups_size":0,"ondisk_log_size":8114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218560,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3050,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4868,"num_write_kb":19472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5058},{"shards":"0,2","objects":3050}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8162","reported_seq":8302,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.562717+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:47.562717+0000","last_peered":"2024-09-16T04:51:47.562717+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:47.562717+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8162,"log_dups_size":0,"ondisk_log_size":8162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33383017,"num_objects":8151,"num_object_clones":0,"num_object_copies":16302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4931,"num_write_kb":19724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4627},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":526381279,"num_objects":128517,"num_object_clones":0,"num_object_copies":257034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3050,"num_objects_unfound":0,"num_objects_dirty":128517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":93751,"num_write_kb":374976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3628,"num_bytes_recovered":14827520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064878080,"data_stored":1064798063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128673,"ondisk_log_size":128673,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":668880,"kb_used_data":402560,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93702960,"statfs":{"total":96636764160,"available":95951831040,"internally_reserved":0,"allocated":412221440,"data_stored":411952457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":272690857},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593755,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":440864,"kb_used_data":234256,"kb_used_omap":4,"kb_used_meta":206587,"kb_avail":93930976,"statfs":{"total":96636764160,"available":96185319424,"internally_reserved":0,"allocated":239878144,"data_stored":239539581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":211545774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":665416,"kb_used_data":403320,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93706424,"statfs":{"total":96636764160,"available":95955378176,"internally_reserved":0,"allocated":412999680,"data_stored":412730697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,7,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":412688384,"data_stored":412684905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":240046080,"data_stored":239973021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":412143616,"data_stored":412140137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:52.310 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:53.112 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 176 20 131263 131243 2.91179 1.86719 0.029209 0.0268196 2024-09-16T04:51:53.206 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:53.207 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:53.273 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-09-16T04:51:52.337554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":531321074,"num_objects":129725,"num_object_clones":0,"num_object_copies":259450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3050,"num_objects_unfound":0,"num_objects_dirty":129725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94959,"num_write_kb":379802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3629,"num_bytes_recovered":14827539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129882,"ondisk_log_size":129882,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1780476,"kb_used_data":1053772,"kb_used_omap":12,"kb_used_meta":726643,"kb_avail":281335044,"statfs":{"total":289910292480,"available":288087085056,"internally_reserved":0,"allocated":1079062528,"data_stored":1078185999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":744082433},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,2,8,14,13,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25223168,"num_objects":6158,"num_object_clones":0,"num_object_copies":12316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-160,"num_objects_unfound":0,"num_objects_dirty":6158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6158,"num_write_kb":24632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.016078"},"pg_stats":[{"pgid":"2.f","version":"47'7889","reported_seq":7967,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.751817+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:47.751817+0000","last_peered":"2024-09-16T04:51:47.751817+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:47.751817+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":7889,"log_dups_size":0,"ondisk_log_size":7889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313344,"num_objects":7889,"num_object_clones":0,"num_object_copies":15778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7889,"num_write_kb":31556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4548},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8160","reported_seq":8276,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:52.044261+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:52.044261+0000","last_peered":"2024-09-16T04:51:52.044261+0000","last_clean":"2024-09-16T04:51:52.044261+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:52.044261+0000","last_undegraded":"2024-09-16T04:51:52.044261+0000","last_fullsized":"2024-09-16T04:51:52.044261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8160,"log_dups_size":0,"ondisk_log_size":8160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33398784,"num_objects":8154,"num_object_clones":0,"num_object_copies":16308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5129,"num_write_kb":20516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8083","reported_seq":8218,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:51.970697+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:51.970697+0000","last_peered":"2024-09-16T04:51:51.970697+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:51.970697+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8083,"log_dups_size":0,"ondisk_log_size":8083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33091584,"num_objects":8079,"num_object_clones":0,"num_object_copies":16158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4822,"num_write_kb":19288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4760}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'8238","reported_seq":8307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:52.070301+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:52.070301+0000","last_peered":"2024-09-16T04:51:52.070301+0000","last_clean":"2024-09-16T04:51:52.070301+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:52.070301+0000","last_undegraded":"2024-09-16T04:51:52.070301+0000","last_fullsized":"2024-09-16T04:51:52.070301+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8238,"log_dups_size":0,"ondisk_log_size":8238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33742848,"num_objects":8238,"num_object_clones":0,"num_object_copies":16476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8238,"num_write_kb":32952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8077","reported_seq":8296,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:47.661113+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:47.661113+0000","last_peered":"2024-09-16T04:51:47.661113+0000","last_clean":"2024-09-16T04:51:47.661113+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:47.661113+0000","last_undegraded":"2024-09-16T04:51:47.661113+0000","last_fullsized":"2024-09-16T04:51:47.661113+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8077,"log_dups_size":0,"ondisk_log_size":8077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960512,"num_objects":8047,"num_object_clones":0,"num_object_copies":16094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5109,"num_write_kb":20436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'8121","reported_seq":8254,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.674584+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:47.674584+0000","last_peered":"2024-09-16T04:51:47.674584+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:47.674584+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":8121,"log_dups_size":0,"ondisk_log_size":8121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33243136,"num_objects":8116,"num_object_clones":0,"num_object_copies":16232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5114,"num_write_kb":20456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4660},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8000","reported_seq":8147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.770850+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:47.770850+0000","last_peered":"2024-09-16T04:51:47.770850+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:47.770850+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8000,"log_dups_size":0,"ondisk_log_size":8000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32743424,"num_objects":7994,"num_object_clones":0,"num_object_copies":15988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4595},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8013","reported_seq":8229,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:47.713625+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:47.713625+0000","last_peered":"2024-09-16T04:51:47.713625+0000","last_clean":"2024-09-16T04:51:47.713625+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:47.713625+0000","last_undegraded":"2024-09-16T04:51:47.713625+0000","last_fullsized":"2024-09-16T04:51:47.713625+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8013,"log_dups_size":0,"ondisk_log_size":8013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32788480,"num_objects":8005,"num_object_clones":0,"num_object_copies":16010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5083,"num_write_kb":20332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8219","reported_seq":8447,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:52.030637+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:52.030637+0000","last_peered":"2024-09-16T04:51:52.030637+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:52.030637+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8219,"log_dups_size":0,"ondisk_log_size":8219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33529856,"num_objects":8186,"num_object_clones":0,"num_object_copies":16372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4970,"num_write_kb":19880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":4865}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8116","reported_seq":8246,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.684890+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:47.684890+0000","last_peered":"2024-09-16T04:51:47.684890+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:47.684890+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8116,"log_dups_size":0,"ondisk_log_size":8116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33210368,"num_objects":8108,"num_object_clones":0,"num_object_copies":16216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4771,"num_write_kb":19084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4643},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8107","reported_seq":8349,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:51.994112+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:51.994112+0000","last_peered":"2024-09-16T04:51:51.994112+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:51.994112+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8107,"log_dups_size":0,"ondisk_log_size":8107,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33080530,"num_objects":8078,"num_object_clones":0,"num_object_copies":16156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8079,"num_write_kb":32307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4782}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8142","reported_seq":14998,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:51.987047+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:51:51.987047+0000","last_peered":"2024-09-16T04:51:51.987047+0000","last_clean":"2024-09-16T04:51:51.987047+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:51:51.987047+0000","last_undegraded":"2024-09-16T04:51:51.987047+0000","last_fullsized":"2024-09-16T04:51:51.987047+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8142,"log_dups_size":0,"ondisk_log_size":8142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4865,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8105","reported_seq":8203,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:52.020738+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:52.020738+0000","last_peered":"2024-09-16T04:51:52.020738+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:52.020738+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8105,"log_dups_size":0,"ondisk_log_size":8105,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33190505,"num_objects":8104,"num_object_clones":0,"num_object_copies":16208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8104,"num_write_kb":32413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4697}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8333","reported_seq":8436,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:51.974464+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:51.974464+0000","last_peered":"2024-09-16T04:51:51.974464+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:51.974464+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8333,"log_dups_size":0,"ondisk_log_size":8333,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34101051,"num_objects":8328,"num_object_clones":0,"num_object_copies":16656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8330,"num_write_kb":33304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4814}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8114","reported_seq":8792,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.764236+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:51:47.764236+0000","last_peered":"2024-09-16T04:51:47.764236+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:47.764236+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8114,"log_dups_size":0,"ondisk_log_size":8114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218560,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":3050,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4868,"num_write_kb":19472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5058},{"shards":"0,2","objects":3050}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8162","reported_seq":8302,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:47.562717+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:47.562717+0000","last_peered":"2024-09-16T04:51:47.562717+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:47.562717+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8162,"log_dups_size":0,"ondisk_log_size":8162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33383017,"num_objects":8151,"num_object_clones":0,"num_object_copies":16302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4931,"num_write_kb":19724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4627},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":531321055,"num_objects":129723,"num_object_clones":0,"num_object_copies":259446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":3050,"num_objects_unfound":0,"num_objects_dirty":129723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94957,"num_write_kb":379800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3628,"num_bytes_recovered":14827520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1078755328,"data_stored":1078675311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129879,"ondisk_log_size":129879,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678988,"kb_used_data":408508,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93692852,"statfs":{"total":96636764160,"available":95941480448,"internally_reserved":0,"allocated":418312192,"data_stored":418043209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":276950697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,1,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593756,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436072,"kb_used_data":241944,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93935768,"statfs":{"total":96636764160,"available":96190226432,"internally_reserved":0,"allocated":247750656,"data_stored":247412093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":198766254},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":665416,"kb_used_data":403320,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93706424,"statfs":{"total":96636764160,"available":95955378176,"internally_reserved":0,"allocated":412999680,"data_stored":412730697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,7,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":412688384,"data_stored":412684905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":247922688,"data_stored":247849629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":418144256,"data_stored":418140777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:53.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:51:53.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'7889", 'reported_seq': 7967, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:47.751817+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:51:47.751817+0000', 'last_peered': '2024-09-16T04:51:47.751817+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:51:47.751817+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 7889, 'log_dups_size': 0, 'ondisk_log_size': 7889, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32313344, 'num_objects': 7889, 'num_object_clones': 0, 'num_object_copies': 15778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7889, 'num_write_kb': 31556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4548}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:53.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:51:53.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'8083", 'reported_seq': 8218, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:51.970697+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:51:51.970697+0000', 'last_peered': '2024-09-16T04:51:51.970697+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:51:51.970697+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 8083, 'log_dups_size': 0, 'ondisk_log_size': 8083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33091584, 'num_objects': 8079, 'num_object_clones': 0, 'num_object_copies': 16158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8079, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4822, 'num_write_kb': 19288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 4760}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:53.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:51:53.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'8121", 'reported_seq': 8254, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:47.674584+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:51:47.674584+0000', 'last_peered': '2024-09-16T04:51:47.674584+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:51:47.674584+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 8121, 'log_dups_size': 0, 'ondisk_log_size': 8121, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33243136, 'num_objects': 8116, 'num_object_clones': 0, 'num_object_copies': 16232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8116, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5114, 'num_write_kb': 20456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4660}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:53.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:51:53.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'8000", 'reported_seq': 8147, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:47.770850+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:51:47.770850+0000', 'last_peered': '2024-09-16T04:51:47.770850+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:51:47.770850+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 8000, 'log_dups_size': 0, 'ondisk_log_size': 8000, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32743424, 'num_objects': 7994, 'num_object_clones': 0, 'num_object_copies': 15988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7994, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4655, 'num_write_kb': 18620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4595}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:53.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:51:53.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'8219", 'reported_seq': 8447, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:52.030637+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:51:52.030637+0000', 'last_peered': '2024-09-16T04:51:52.030637+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:51:52.030637+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 8219, 'log_dups_size': 0, 'ondisk_log_size': 8219, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33529856, 'num_objects': 8186, 'num_object_clones': 0, 'num_object_copies': 16372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8186, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4970, 'num_write_kb': 19880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 4865}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:53.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:51:53.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'8116", 'reported_seq': 8246, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:47.684890+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:51:47.684890+0000', 'last_peered': '2024-09-16T04:51:47.684890+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:51:47.684890+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 8116, 'log_dups_size': 0, 'ondisk_log_size': 8116, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33210368, 'num_objects': 8108, 'num_object_clones': 0, 'num_object_copies': 16216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8108, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4771, 'num_write_kb': 19084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4643}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:53.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:51:53.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'8107", 'reported_seq': 8349, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:51.994112+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:51:51.994112+0000', 'last_peered': '2024-09-16T04:51:51.994112+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:51:51.994112+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 8107, 'log_dups_size': 0, 'ondisk_log_size': 8107, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33080530, 'num_objects': 8078, 'num_object_clones': 0, 'num_object_copies': 16156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8078, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8079, 'num_write_kb': 32307, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 4782}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:53.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:51:53.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'8105", 'reported_seq': 8203, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:52.020738+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:51:52.020738+0000', 'last_peered': '2024-09-16T04:51:52.020738+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:51:52.020738+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 8105, 'log_dups_size': 0, 'ondisk_log_size': 8105, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33190505, 'num_objects': 8104, 'num_object_clones': 0, 'num_object_copies': 16208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8104, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8104, 'num_write_kb': 32413, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 4697}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:53.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:51:53.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'8333", 'reported_seq': 8436, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:51.974464+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:51:51.974464+0000', 'last_peered': '2024-09-16T04:51:51.974464+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:51:51.974464+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 8333, 'log_dups_size': 0, 'ondisk_log_size': 8333, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34101051, 'num_objects': 8328, 'num_object_clones': 0, 'num_object_copies': 16656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8328, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8330, 'num_write_kb': 33304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 4814}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:51:53.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:51:53.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'8114", 'reported_seq': 8792, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:47.764236+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:51:47.764236+0000', 'last_peered': '2024-09-16T04:51:47.764236+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:51:47.764236+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 8114, 'log_dups_size': 0, 'ondisk_log_size': 8114, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33218560, 'num_objects': 8110, 'num_object_clones': 0, 'num_object_copies': 16220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 3050, 'num_objects_unfound': 0, 'num_objects_dirty': 8110, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4868, 'num_write_kb': 19472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 273, 'num_bytes_recovered': 1118208, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5058}, {'shards': '0,2', 'objects': 3050}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:53.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:51:53.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'8162", 'reported_seq': 8302, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:47.562717+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:51:47.562717+0000', 'last_peered': '2024-09-16T04:51:47.562717+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:51:47.562717+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 8162, 'log_dups_size': 0, 'ondisk_log_size': 8162, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33383017, 'num_objects': 8151, 'num_object_clones': 0, 'num_object_copies': 16302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8151, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4931, 'num_write_kb': 19724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4627}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:51:53.277 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:51:54.100 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:54.101 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/259450 objects degraded (13.129%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":261,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":129725,"data_bytes":531321074,"bytes_used":1823207424,"bytes_avail":288087085056,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":259450,"degraded_ratio":0.13128541144729236,"misplaced_objects":3050,"misplaced_total":259450,"misplaced_ratio":0.011755636924262864,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":54540,"recovering_keys_per_sec":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"write_bytes_sec":2099118,"read_op_per_sec":0,"write_op_per_sec":512},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:51:54.112 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 177 20 131748 131728 2.90605 1.89453 0.0686562 0.0268754 2024-09-16T04:51:54.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:51:54.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:54.989 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:55.043 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-16T04:51:54.340792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":536297714,"num_objects":130940,"num_object_clones":0,"num_object_copies":261880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2964,"num_objects_unfound":0,"num_objects_dirty":130940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96174,"num_write_kb":384662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3715,"num_bytes_recovered":15179795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131097,"ondisk_log_size":131097,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1794880,"kb_used_data":1059856,"kb_used_omap":12,"kb_used_meta":734963,"kb_avail":281320640,"statfs":{"total":289910292480,"available":288072335360,"internally_reserved":0,"allocated":1085292544,"data_stored":1084416015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":752602113},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,2,4,14,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25018368,"num_objects":6108,"num_object_clones":0,"num_object_copies":12216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-246,"num_objects_unfound":0,"num_objects_dirty":6108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6108,"num_write_kb":24432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1007616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.018863"},"pg_stats":[{"pgid":"2.f","version":"47'8056","reported_seq":8132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:52.821180+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:52.821180+0000","last_peered":"2024-09-16T04:51:52.821180+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:52.821180+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8056,"log_dups_size":0,"ondisk_log_size":8056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32997376,"num_objects":8056,"num_object_clones":0,"num_object_copies":16112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8056,"num_write_kb":32224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4715},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8160","reported_seq":8276,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:52.044261+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:52.044261+0000","last_peered":"2024-09-16T04:51:52.044261+0000","last_clean":"2024-09-16T04:51:52.044261+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:52.044261+0000","last_undegraded":"2024-09-16T04:51:52.044261+0000","last_fullsized":"2024-09-16T04:51:52.044261+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8160,"log_dups_size":0,"ondisk_log_size":8160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33398784,"num_objects":8154,"num_object_clones":0,"num_object_copies":16308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5129,"num_write_kb":20516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8083","reported_seq":8218,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:51.970697+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:51.970697+0000","last_peered":"2024-09-16T04:51:51.970697+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:51.970697+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8083,"log_dups_size":0,"ondisk_log_size":8083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33091584,"num_objects":8079,"num_object_clones":0,"num_object_copies":16158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4822,"num_write_kb":19288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4760}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'8238","reported_seq":8307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:52.070301+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:52.070301+0000","last_peered":"2024-09-16T04:51:52.070301+0000","last_clean":"2024-09-16T04:51:52.070301+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:52.070301+0000","last_undegraded":"2024-09-16T04:51:52.070301+0000","last_fullsized":"2024-09-16T04:51:52.070301+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8238,"log_dups_size":0,"ondisk_log_size":8238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33742848,"num_objects":8238,"num_object_clones":0,"num_object_copies":16476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8238,"num_write_kb":32952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8203","reported_seq":8421,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:52.801570+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:52.801570+0000","last_peered":"2024-09-16T04:51:52.801570+0000","last_clean":"2024-09-16T04:51:52.801570+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:52.801570+0000","last_undegraded":"2024-09-16T04:51:52.801570+0000","last_fullsized":"2024-09-16T04:51:52.801570+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8203,"log_dups_size":0,"ondisk_log_size":8203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33476608,"num_objects":8173,"num_object_clones":0,"num_object_copies":16346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5235,"num_write_kb":20940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'8292","reported_seq":8425,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:52.718419+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:52.718419+0000","last_peered":"2024-09-16T04:51:52.718419+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:52.718419+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":8292,"log_dups_size":0,"ondisk_log_size":8292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33943552,"num_objects":8287,"num_object_clones":0,"num_object_copies":16574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5285,"num_write_kb":21140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4831},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8162","reported_seq":8310,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:52.754942+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:52.754942+0000","last_peered":"2024-09-16T04:51:52.754942+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:52.754942+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8162,"log_dups_size":0,"ondisk_log_size":8162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33406976,"num_objects":8156,"num_object_clones":0,"num_object_copies":16312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4817,"num_write_kb":19268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4757},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8160","reported_seq":8376,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:52.736994+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:52.736994+0000","last_peered":"2024-09-16T04:51:52.736994+0000","last_clean":"2024-09-16T04:51:52.736994+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:52.736994+0000","last_undegraded":"2024-09-16T04:51:52.736994+0000","last_fullsized":"2024-09-16T04:51:52.736994+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8160,"log_dups_size":0,"ondisk_log_size":8160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33390592,"num_objects":8152,"num_object_clones":0,"num_object_copies":16304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5230,"num_write_kb":20920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8219","reported_seq":8447,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:52.030637+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:52.030637+0000","last_peered":"2024-09-16T04:51:52.030637+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:52.030637+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8219,"log_dups_size":0,"ondisk_log_size":8219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33529856,"num_objects":8186,"num_object_clones":0,"num_object_copies":16372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4970,"num_write_kb":19880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":4865}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8255","reported_seq":8385,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:52.777340+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:52.777340+0000","last_peered":"2024-09-16T04:51:52.777340+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:52.777340+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8255,"log_dups_size":0,"ondisk_log_size":8255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33779712,"num_objects":8247,"num_object_clones":0,"num_object_copies":16494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4910,"num_write_kb":19640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4782},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8107","reported_seq":8349,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:51.994112+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:51.994112+0000","last_peered":"2024-09-16T04:51:51.994112+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:51.994112+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8107,"log_dups_size":0,"ondisk_log_size":8107,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33080530,"num_objects":8078,"num_object_clones":0,"num_object_copies":16156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8079,"num_write_kb":32307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4782}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8142","reported_seq":14998,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:51.987047+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:51:51.987047+0000","last_peered":"2024-09-16T04:51:51.987047+0000","last_clean":"2024-09-16T04:51:51.987047+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:51:51.987047+0000","last_undegraded":"2024-09-16T04:51:51.987047+0000","last_fullsized":"2024-09-16T04:51:51.987047+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8142,"log_dups_size":0,"ondisk_log_size":8142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4865,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8105","reported_seq":8203,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:52.020738+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:52.020738+0000","last_peered":"2024-09-16T04:51:52.020738+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:52.020738+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8105,"log_dups_size":0,"ondisk_log_size":8105,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33190505,"num_objects":8104,"num_object_clones":0,"num_object_copies":16208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8104,"num_write_kb":32413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4697}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8333","reported_seq":8436,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:51.974464+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:51.974464+0000","last_peered":"2024-09-16T04:51:51.974464+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:51.974464+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8333,"log_dups_size":0,"ondisk_log_size":8333,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34101051,"num_objects":8328,"num_object_clones":0,"num_object_copies":16656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8330,"num_write_kb":33304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4814}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8271","reported_seq":9121,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:52.805054+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:51:52.805054+0000","last_peered":"2024-09-16T04:51:52.805054+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:52.805054+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8271,"log_dups_size":0,"ondisk_log_size":8271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33861632,"num_objects":8267,"num_object_clones":0,"num_object_copies":16534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2964,"num_objects_unfound":0,"num_objects_dirty":8267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5025,"num_write_kb":20100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":1470464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5301},{"shards":"0,2","objects":2964}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8308","reported_seq":8448,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:52.817931+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:52.817931+0000","last_peered":"2024-09-16T04:51:52.817931+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:52.817931+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8308,"log_dups_size":0,"ondisk_log_size":8308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33981033,"num_objects":8297,"num_object_clones":0,"num_object_copies":16594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5077,"num_write_kb":20308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4773},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":536297695,"num_objects":130938,"num_object_clones":0,"num_object_copies":261876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2964,"num_objects_unfound":0,"num_objects_dirty":130938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96172,"num_write_kb":384660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3714,"num_bytes_recovered":15179776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1084997632,"data_stored":1084917615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131094,"ondisk_log_size":131094,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678988,"kb_used_data":408508,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93692852,"statfs":{"total":96636764160,"available":95941480448,"internally_reserved":0,"allocated":418312192,"data_stored":418043209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":276950697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,1,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593756,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436072,"kb_used_data":241944,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93935768,"statfs":{"total":96636764160,"available":96190226432,"internally_reserved":0,"allocated":247750656,"data_stored":247412093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":198766254},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679820,"kb_used_data":409404,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93692020,"statfs":{"total":96636764160,"available":95940628480,"internally_reserved":0,"allocated":419229696,"data_stored":418960713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,3,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":418930688,"data_stored":418927209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":247922688,"data_stored":247849629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":418144256,"data_stored":418140777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:55.112 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 178 20 132227 132207 2.90023 1.87109 0.04781 0.0269308 2024-09-16T04:51:56.112 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 179 20 132728 132708 2.89496 1.95703 0.0231215 0.0269789 2024-09-16T04:51:57.117 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:2024-09-16T04:51:57.116742+0000 min lat: 0.00206892 max lat: 2.69414 avg lat: 0.0270265 2024-09-16T04:51:57.117 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T04:51:57.117 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 180 20 133236 133216 2.88982 1.98438 0.0310434 0.0270265 2024-09-16T04:51:58.045 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:58.117 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 181 20 133742 133722 2.88478 1.97656 0.0516506 0.0270747 2024-09-16T04:51:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:59.061 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:51:59.118 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 182 20 134219 134199 2.87916 1.86328 0.0170314 0.0271251 2024-09-16T04:51:59.125 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-16T04:51:58.343175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":546459890,"num_objects":133421,"num_object_clones":0,"num_object_copies":266842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2881,"num_objects_unfound":0,"num_objects_dirty":133421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98655,"num_write_kb":394586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3798,"num_bytes_recovered":15519763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133578,"ondisk_log_size":133578,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1835704,"kb_used_data":1079880,"kb_used_omap":12,"kb_used_meta":755763,"kb_avail":281279816,"statfs":{"total":289910292480,"available":288030531584,"internally_reserved":0,"allocated":1105797120,"data_stored":1104920591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":773901313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,7,8,7,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30236672,"num_objects":7382,"num_object_clones":0,"num_object_copies":14764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":7382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7382,"num_write_kb":29528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009841"},"pg_stats":[{"pgid":"2.f","version":"47'8216","reported_seq":8295,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.828623+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:57.828623+0000","last_peered":"2024-09-16T04:51:57.828623+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:57.828623+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8216,"log_dups_size":0,"ondisk_log_size":8216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652736,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8216,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4875},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8318","reported_seq":8434,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:56.992330+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:56.992330+0000","last_peered":"2024-09-16T04:51:56.992330+0000","last_clean":"2024-09-16T04:51:56.992330+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:56.992330+0000","last_undegraded":"2024-09-16T04:51:56.992330+0000","last_fullsized":"2024-09-16T04:51:56.992330+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8318,"log_dups_size":0,"ondisk_log_size":8318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5287,"num_write_kb":21148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8248","reported_seq":8386,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.085533+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:57.085533+0000","last_peered":"2024-09-16T04:51:57.085533+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:57.085533+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8248,"log_dups_size":0,"ondisk_log_size":8248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33767424,"num_objects":8244,"num_object_clones":0,"num_object_copies":16488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4987,"num_write_kb":19948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4925}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'8391","reported_seq":8461,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:57.028490+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:57.028490+0000","last_peered":"2024-09-16T04:51:57.028490+0000","last_clean":"2024-09-16T04:51:57.028490+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:57.028490+0000","last_undegraded":"2024-09-16T04:51:57.028490+0000","last_fullsized":"2024-09-16T04:51:57.028490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8391,"log_dups_size":0,"ondisk_log_size":8391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34369536,"num_objects":8391,"num_object_clones":0,"num_object_copies":16782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8391,"num_write_kb":33564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8366","reported_seq":8585,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:57.832920+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:57.832920+0000","last_peered":"2024-09-16T04:51:57.832920+0000","last_clean":"2024-09-16T04:51:57.832920+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:57.832920+0000","last_undegraded":"2024-09-16T04:51:57.832920+0000","last_fullsized":"2024-09-16T04:51:57.832920+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8366,"log_dups_size":0,"ondisk_log_size":8366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34144256,"num_objects":8336,"num_object_clones":0,"num_object_copies":16672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5398,"num_write_kb":21592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'8433","reported_seq":8563,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.774116+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:57.774116+0000","last_peered":"2024-09-16T04:51:57.774116+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:57.774116+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":8433,"log_dups_size":0,"ondisk_log_size":8433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34521088,"num_objects":8428,"num_object_clones":0,"num_object_copies":16856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5426,"num_write_kb":21704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4972},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8328","reported_seq":8476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.855140+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:57.855140+0000","last_peered":"2024-09-16T04:51:57.855140+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:57.855140+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8328,"log_dups_size":0,"ondisk_log_size":8328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34086912,"num_objects":8322,"num_object_clones":0,"num_object_copies":16644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4983,"num_write_kb":19932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4923},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8325","reported_seq":8541,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:57.845047+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:57.845047+0000","last_peered":"2024-09-16T04:51:57.845047+0000","last_clean":"2024-09-16T04:51:57.845047+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:57.845047+0000","last_undegraded":"2024-09-16T04:51:57.845047+0000","last_fullsized":"2024-09-16T04:51:57.845047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8325,"log_dups_size":0,"ondisk_log_size":8325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34066432,"num_objects":8317,"num_object_clones":0,"num_object_copies":16634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5395,"num_write_kb":21580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8389","reported_seq":8617,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.016311+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:57.016311+0000","last_peered":"2024-09-16T04:51:57.016311+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:57.016311+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8389,"log_dups_size":0,"ondisk_log_size":8389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34226176,"num_objects":8356,"num_object_clones":0,"num_object_copies":16712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5140,"num_write_kb":20560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5035}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8389","reported_seq":8519,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.797334+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:57.797334+0000","last_peered":"2024-09-16T04:51:57.797334+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:57.797334+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8389,"log_dups_size":0,"ondisk_log_size":8389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34328576,"num_objects":8381,"num_object_clones":0,"num_object_copies":16762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5044,"num_write_kb":20176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4916},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8255","reported_seq":8497,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.052975+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:57.052975+0000","last_peered":"2024-09-16T04:51:57.052975+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:57.052975+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8255,"log_dups_size":0,"ondisk_log_size":8255,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33686738,"num_objects":8226,"num_object_clones":0,"num_object_copies":16452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8227,"num_write_kb":32899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4930}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8305","reported_seq":15161,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:57.033678+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:51:57.033678+0000","last_peered":"2024-09-16T04:51:57.033678+0000","last_clean":"2024-09-16T04:51:57.033678+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:51:57.033678+0000","last_undegraded":"2024-09-16T04:51:57.033678+0000","last_fullsized":"2024-09-16T04:51:57.033678+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8305,"log_dups_size":0,"ondisk_log_size":8305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33992704,"num_objects":8299,"num_object_clones":0,"num_object_copies":16598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5028,"num_write_kb":20112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8260","reported_seq":8358,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.075703+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:57.075703+0000","last_peered":"2024-09-16T04:51:57.075703+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:57.075703+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8260,"log_dups_size":0,"ondisk_log_size":8260,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33825385,"num_objects":8259,"num_object_clones":0,"num_object_copies":16518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8259,"num_write_kb":33033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4852}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8473","reported_seq":8575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.058071+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:57.058071+0000","last_peered":"2024-09-16T04:51:57.058071+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:57.058071+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8473,"log_dups_size":0,"ondisk_log_size":8473,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34674491,"num_objects":8468,"num_object_clones":0,"num_object_copies":16936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8470,"num_write_kb":33864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4954}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8403","reported_seq":9419,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.847378+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:51:57.847378+0000","last_peered":"2024-09-16T04:51:57.847378+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:57.847378+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8403,"log_dups_size":0,"ondisk_log_size":8403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402304,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2881,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5157,"num_write_kb":20628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":1810432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5516},{"shards":"0,2","objects":2881}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8476","reported_seq":8616,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.808193+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:57.808193+0000","last_peered":"2024-09-16T04:51:57.808193+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:57.808193+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8476,"log_dups_size":0,"ondisk_log_size":8476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34669161,"num_objects":8465,"num_object_clones":0,"num_object_copies":16930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5245,"num_write_kb":20980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4941},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":546459871,"num_objects":133419,"num_object_clones":0,"num_object_copies":266838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2881,"num_objects_unfound":0,"num_objects_dirty":133419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98653,"num_write_kb":394584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3797,"num_bytes_recovered":15519744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1105563648,"data_stored":1105483631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133575,"ondisk_log_size":133575,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693672,"kb_used_data":414872,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93678168,"statfs":{"total":96636764160,"available":95926444032,"internally_reserved":0,"allocated":424828928,"data_stored":424559945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":285470377},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593757,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451924,"kb_used_data":249476,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93919916,"statfs":{"total":96636764160,"available":96173993984,"internally_reserved":0,"allocated":255463424,"data_stored":255124861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207285934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":690108,"kb_used_data":415532,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93681732,"statfs":{"total":96636764160,"available":95930093568,"internally_reserved":0,"allocated":425504768,"data_stored":425235785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,2,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":425267200,"data_stored":425263721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255688704,"data_stored":255615645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":424607744,"data_stored":424604265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:51:59.126 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:51:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:51:59.971 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:00.049 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-16T04:51:58.343175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":546459890,"num_objects":133421,"num_object_clones":0,"num_object_copies":266842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2881,"num_objects_unfound":0,"num_objects_dirty":133421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98655,"num_write_kb":394586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3798,"num_bytes_recovered":15519763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133578,"ondisk_log_size":133578,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1835704,"kb_used_data":1079880,"kb_used_omap":12,"kb_used_meta":755763,"kb_avail":281279816,"statfs":{"total":289910292480,"available":288030531584,"internally_reserved":0,"allocated":1105797120,"data_stored":1104920591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":773901313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,7,8,7,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30236672,"num_objects":7382,"num_object_clones":0,"num_object_copies":14764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":7382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7382,"num_write_kb":29528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009841"},"pg_stats":[{"pgid":"2.f","version":"47'8216","reported_seq":8295,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.828623+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:57.828623+0000","last_peered":"2024-09-16T04:51:57.828623+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:57.828623+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8216,"log_dups_size":0,"ondisk_log_size":8216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652736,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8216,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4875},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8318","reported_seq":8434,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:56.992330+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:56.992330+0000","last_peered":"2024-09-16T04:51:56.992330+0000","last_clean":"2024-09-16T04:51:56.992330+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:56.992330+0000","last_undegraded":"2024-09-16T04:51:56.992330+0000","last_fullsized":"2024-09-16T04:51:56.992330+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8318,"log_dups_size":0,"ondisk_log_size":8318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5287,"num_write_kb":21148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8248","reported_seq":8386,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.085533+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:57.085533+0000","last_peered":"2024-09-16T04:51:57.085533+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:57.085533+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8248,"log_dups_size":0,"ondisk_log_size":8248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33767424,"num_objects":8244,"num_object_clones":0,"num_object_copies":16488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4987,"num_write_kb":19948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4925}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'8391","reported_seq":8461,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:57.028490+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:57.028490+0000","last_peered":"2024-09-16T04:51:57.028490+0000","last_clean":"2024-09-16T04:51:57.028490+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:57.028490+0000","last_undegraded":"2024-09-16T04:51:57.028490+0000","last_fullsized":"2024-09-16T04:51:57.028490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8391,"log_dups_size":0,"ondisk_log_size":8391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34369536,"num_objects":8391,"num_object_clones":0,"num_object_copies":16782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8391,"num_write_kb":33564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8366","reported_seq":8585,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:57.832920+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:57.832920+0000","last_peered":"2024-09-16T04:51:57.832920+0000","last_clean":"2024-09-16T04:51:57.832920+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:57.832920+0000","last_undegraded":"2024-09-16T04:51:57.832920+0000","last_fullsized":"2024-09-16T04:51:57.832920+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8366,"log_dups_size":0,"ondisk_log_size":8366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34144256,"num_objects":8336,"num_object_clones":0,"num_object_copies":16672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5398,"num_write_kb":21592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'8433","reported_seq":8563,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.774116+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:57.774116+0000","last_peered":"2024-09-16T04:51:57.774116+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:57.774116+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":8433,"log_dups_size":0,"ondisk_log_size":8433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34521088,"num_objects":8428,"num_object_clones":0,"num_object_copies":16856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5426,"num_write_kb":21704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4972},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8328","reported_seq":8476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.855140+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:57.855140+0000","last_peered":"2024-09-16T04:51:57.855140+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:57.855140+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8328,"log_dups_size":0,"ondisk_log_size":8328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34086912,"num_objects":8322,"num_object_clones":0,"num_object_copies":16644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4983,"num_write_kb":19932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4923},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8325","reported_seq":8541,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:57.845047+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:57.845047+0000","last_peered":"2024-09-16T04:51:57.845047+0000","last_clean":"2024-09-16T04:51:57.845047+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:57.845047+0000","last_undegraded":"2024-09-16T04:51:57.845047+0000","last_fullsized":"2024-09-16T04:51:57.845047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8325,"log_dups_size":0,"ondisk_log_size":8325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34066432,"num_objects":8317,"num_object_clones":0,"num_object_copies":16634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5395,"num_write_kb":21580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8389","reported_seq":8617,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.016311+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:57.016311+0000","last_peered":"2024-09-16T04:51:57.016311+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:57.016311+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8389,"log_dups_size":0,"ondisk_log_size":8389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34226176,"num_objects":8356,"num_object_clones":0,"num_object_copies":16712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5140,"num_write_kb":20560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5035}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8389","reported_seq":8519,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.797334+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:57.797334+0000","last_peered":"2024-09-16T04:51:57.797334+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:57.797334+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8389,"log_dups_size":0,"ondisk_log_size":8389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34328576,"num_objects":8381,"num_object_clones":0,"num_object_copies":16762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5044,"num_write_kb":20176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4916},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8255","reported_seq":8497,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.052975+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:57.052975+0000","last_peered":"2024-09-16T04:51:57.052975+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:57.052975+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8255,"log_dups_size":0,"ondisk_log_size":8255,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33686738,"num_objects":8226,"num_object_clones":0,"num_object_copies":16452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8227,"num_write_kb":32899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4930}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8305","reported_seq":15161,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:57.033678+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:51:57.033678+0000","last_peered":"2024-09-16T04:51:57.033678+0000","last_clean":"2024-09-16T04:51:57.033678+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:51:57.033678+0000","last_undegraded":"2024-09-16T04:51:57.033678+0000","last_fullsized":"2024-09-16T04:51:57.033678+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8305,"log_dups_size":0,"ondisk_log_size":8305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33992704,"num_objects":8299,"num_object_clones":0,"num_object_copies":16598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5028,"num_write_kb":20112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8260","reported_seq":8358,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.075703+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:57.075703+0000","last_peered":"2024-09-16T04:51:57.075703+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:57.075703+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8260,"log_dups_size":0,"ondisk_log_size":8260,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33825385,"num_objects":8259,"num_object_clones":0,"num_object_copies":16518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8259,"num_write_kb":33033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4852}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8473","reported_seq":8575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.058071+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:57.058071+0000","last_peered":"2024-09-16T04:51:57.058071+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:57.058071+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8473,"log_dups_size":0,"ondisk_log_size":8473,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34674491,"num_objects":8468,"num_object_clones":0,"num_object_copies":16936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8470,"num_write_kb":33864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4954}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8403","reported_seq":9419,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.847378+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:51:57.847378+0000","last_peered":"2024-09-16T04:51:57.847378+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:57.847378+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8403,"log_dups_size":0,"ondisk_log_size":8403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402304,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2881,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5157,"num_write_kb":20628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":1810432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5516},{"shards":"0,2","objects":2881}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8476","reported_seq":8616,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.808193+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:57.808193+0000","last_peered":"2024-09-16T04:51:57.808193+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:57.808193+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8476,"log_dups_size":0,"ondisk_log_size":8476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34669161,"num_objects":8465,"num_object_clones":0,"num_object_copies":16930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5245,"num_write_kb":20980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4941},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":546459871,"num_objects":133419,"num_object_clones":0,"num_object_copies":266838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2881,"num_objects_unfound":0,"num_objects_dirty":133419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98653,"num_write_kb":394584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3797,"num_bytes_recovered":15519744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1105563648,"data_stored":1105483631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133575,"ondisk_log_size":133575,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693672,"kb_used_data":414872,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93678168,"statfs":{"total":96636764160,"available":95926444032,"internally_reserved":0,"allocated":424828928,"data_stored":424559945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":285470377},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593757,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451924,"kb_used_data":249476,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93919916,"statfs":{"total":96636764160,"available":96173993984,"internally_reserved":0,"allocated":255463424,"data_stored":255124861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207285934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":690108,"kb_used_data":415532,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93681732,"statfs":{"total":96636764160,"available":95930093568,"internally_reserved":0,"allocated":425504768,"data_stored":425235785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,2,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":425267200,"data_stored":425263721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255688704,"data_stored":255615645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":424607744,"data_stored":424604265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:00.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:52:00.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'8216", 'reported_seq': 8295, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:57.828623+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:51:57.828623+0000', 'last_peered': '2024-09-16T04:51:57.828623+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:51:57.828623+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 8216, 'log_dups_size': 0, 'ondisk_log_size': 8216, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33652736, 'num_objects': 8216, 'num_object_clones': 0, 'num_object_copies': 16432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8216, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8216, 'num_write_kb': 32864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4875}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:00.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:52:00.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'8248", 'reported_seq': 8386, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:57.085533+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:51:57.085533+0000', 'last_peered': '2024-09-16T04:51:57.085533+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:51:57.085533+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 8248, 'log_dups_size': 0, 'ondisk_log_size': 8248, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33767424, 'num_objects': 8244, 'num_object_clones': 0, 'num_object_copies': 16488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8244, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4987, 'num_write_kb': 19948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 4925}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:00.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:52:00.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'8433", 'reported_seq': 8563, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:57.774116+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:51:57.774116+0000', 'last_peered': '2024-09-16T04:51:57.774116+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:51:57.774116+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 8433, 'log_dups_size': 0, 'ondisk_log_size': 8433, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34521088, 'num_objects': 8428, 'num_object_clones': 0, 'num_object_copies': 16856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5426, 'num_write_kb': 21704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4972}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:00.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:52:00.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'8328", 'reported_seq': 8476, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:57.855140+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:51:57.855140+0000', 'last_peered': '2024-09-16T04:51:57.855140+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:51:57.855140+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 8328, 'log_dups_size': 0, 'ondisk_log_size': 8328, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34086912, 'num_objects': 8322, 'num_object_clones': 0, 'num_object_copies': 16644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8322, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4983, 'num_write_kb': 19932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4923}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:00.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:52:00.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'8389", 'reported_seq': 8617, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:57.016311+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:51:57.016311+0000', 'last_peered': '2024-09-16T04:51:57.016311+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:51:57.016311+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 8389, 'log_dups_size': 0, 'ondisk_log_size': 8389, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34226176, 'num_objects': 8356, 'num_object_clones': 0, 'num_object_copies': 16712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8356, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5140, 'num_write_kb': 20560, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 5035}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:00.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:52:00.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'8389", 'reported_seq': 8519, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:57.797334+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:51:57.797334+0000', 'last_peered': '2024-09-16T04:51:57.797334+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:51:57.797334+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 8389, 'log_dups_size': 0, 'ondisk_log_size': 8389, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34328576, 'num_objects': 8381, 'num_object_clones': 0, 'num_object_copies': 16762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8381, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5044, 'num_write_kb': 20176, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4916}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:00.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:52:00.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'8255", 'reported_seq': 8497, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:57.052975+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:51:57.052975+0000', 'last_peered': '2024-09-16T04:51:57.052975+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:51:57.052975+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 8255, 'log_dups_size': 0, 'ondisk_log_size': 8255, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33686738, 'num_objects': 8226, 'num_object_clones': 0, 'num_object_copies': 16452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8226, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8227, 'num_write_kb': 32899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 4930}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:00.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:52:00.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'8260", 'reported_seq': 8358, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:57.075703+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:51:57.075703+0000', 'last_peered': '2024-09-16T04:51:57.075703+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:51:57.075703+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 8260, 'log_dups_size': 0, 'ondisk_log_size': 8260, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33825385, 'num_objects': 8259, 'num_object_clones': 0, 'num_object_copies': 16518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8259, 'num_write_kb': 33033, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 4852}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:00.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:52:00.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'8473", 'reported_seq': 8575, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:57.058071+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:51:57.058071+0000', 'last_peered': '2024-09-16T04:51:57.058071+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:51:57.058071+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 8473, 'log_dups_size': 0, 'ondisk_log_size': 8473, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34674491, 'num_objects': 8468, 'num_object_clones': 0, 'num_object_copies': 16936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8468, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8470, 'num_write_kb': 33864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 4954}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:00.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:52:00.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'8403", 'reported_seq': 9419, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:57.847378+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:51:57.847378+0000', 'last_peered': '2024-09-16T04:51:57.847378+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:51:57.847378+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 8403, 'log_dups_size': 0, 'ondisk_log_size': 8403, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34402304, 'num_objects': 8399, 'num_object_clones': 0, 'num_object_copies': 16798, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2881, 'num_objects_unfound': 0, 'num_objects_dirty': 8399, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5157, 'num_write_kb': 20628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 442, 'num_bytes_recovered': 1810432, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5516}, {'shards': '0,2', 'objects': 2881}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:00.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:52:00.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'8476", 'reported_seq': 8616, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:51:57.808193+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:51:57.808193+0000', 'last_peered': '2024-09-16T04:51:57.808193+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:51:57.808193+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 8476, 'log_dups_size': 0, 'ondisk_log_size': 8476, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34669161, 'num_objects': 8465, 'num_object_clones': 0, 'num_object_copies': 16930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8465, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5245, 'num_write_kb': 20980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4941}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:00.053 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:52:00.121 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 183 20 134695 134675 2.87359 1.85938 0.0245996 0.0271788 2024-09-16T04:52:00.887 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:00.887 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/266842 objects degraded (12.765%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":268,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":133421,"data_bytes":546459890,"bytes_used":1879760896,"bytes_avail":288030531584,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":266842,"degraded_ratio":0.12764857106452507,"misplaced_objects":2881,"misplaced_total":266842,"misplaced_ratio":0.010796651201834793,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":57642,"recovering_keys_per_sec":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"write_bytes_sec":1671975,"read_op_per_sec":0,"write_op_per_sec":408},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:52:00.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:52:00.936 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:01.118 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 184 20 135128 135108 2.86717 1.69141 0.027819 0.0272404 2024-09-16T04:52:01.658 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:01.712 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-16T04:52:00.345474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":546459890,"num_objects":133421,"num_object_clones":0,"num_object_copies":266842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2881,"num_objects_unfound":0,"num_objects_dirty":133421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98655,"num_write_kb":394586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3798,"num_bytes_recovered":15519763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133578,"ondisk_log_size":133578,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1835704,"kb_used_data":1079880,"kb_used_omap":12,"kb_used_meta":755763,"kb_avail":281279816,"statfs":{"total":289910292480,"available":288030531584,"internally_reserved":0,"allocated":1105797120,"data_stored":1104920591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":773901313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,7,8,7,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20078592,"num_objects":4902,"num_object_clones":0,"num_object_copies":9804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-169,"num_objects_unfound":0,"num_objects_dirty":4902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4902,"num_write_kb":19608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008903"},"pg_stats":[{"pgid":"2.f","version":"47'8216","reported_seq":8295,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.828623+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:51:57.828623+0000","last_peered":"2024-09-16T04:51:57.828623+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:51:57.828623+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8216,"log_dups_size":0,"ondisk_log_size":8216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652736,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8216,"num_write_kb":32864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4875},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8318","reported_seq":8434,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:56.992330+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:51:56.992330+0000","last_peered":"2024-09-16T04:51:56.992330+0000","last_clean":"2024-09-16T04:51:56.992330+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:51:56.992330+0000","last_undegraded":"2024-09-16T04:51:56.992330+0000","last_fullsized":"2024-09-16T04:51:56.992330+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8318,"log_dups_size":0,"ondisk_log_size":8318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34045952,"num_objects":8312,"num_object_clones":0,"num_object_copies":16624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5287,"num_write_kb":21148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8248","reported_seq":8386,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.085533+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:51:57.085533+0000","last_peered":"2024-09-16T04:51:57.085533+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:51:57.085533+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8248,"log_dups_size":0,"ondisk_log_size":8248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33767424,"num_objects":8244,"num_object_clones":0,"num_object_copies":16488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4987,"num_write_kb":19948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":4925}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'8391","reported_seq":8461,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:57.028490+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:51:57.028490+0000","last_peered":"2024-09-16T04:51:57.028490+0000","last_clean":"2024-09-16T04:51:57.028490+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:51:57.028490+0000","last_undegraded":"2024-09-16T04:51:57.028490+0000","last_fullsized":"2024-09-16T04:51:57.028490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8391,"log_dups_size":0,"ondisk_log_size":8391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34369536,"num_objects":8391,"num_object_clones":0,"num_object_copies":16782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8391,"num_write_kb":33564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8366","reported_seq":8585,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:57.832920+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:51:57.832920+0000","last_peered":"2024-09-16T04:51:57.832920+0000","last_clean":"2024-09-16T04:51:57.832920+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:51:57.832920+0000","last_undegraded":"2024-09-16T04:51:57.832920+0000","last_fullsized":"2024-09-16T04:51:57.832920+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8366,"log_dups_size":0,"ondisk_log_size":8366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34144256,"num_objects":8336,"num_object_clones":0,"num_object_copies":16672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5398,"num_write_kb":21592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'8433","reported_seq":8563,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.774116+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:51:57.774116+0000","last_peered":"2024-09-16T04:51:57.774116+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:51:57.774116+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":8433,"log_dups_size":0,"ondisk_log_size":8433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34521088,"num_objects":8428,"num_object_clones":0,"num_object_copies":16856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5426,"num_write_kb":21704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4972},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8328","reported_seq":8476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.855140+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:51:57.855140+0000","last_peered":"2024-09-16T04:51:57.855140+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:51:57.855140+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8328,"log_dups_size":0,"ondisk_log_size":8328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34086912,"num_objects":8322,"num_object_clones":0,"num_object_copies":16644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4983,"num_write_kb":19932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4923},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8325","reported_seq":8541,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:57.845047+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:51:57.845047+0000","last_peered":"2024-09-16T04:51:57.845047+0000","last_clean":"2024-09-16T04:51:57.845047+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:51:57.845047+0000","last_undegraded":"2024-09-16T04:51:57.845047+0000","last_fullsized":"2024-09-16T04:51:57.845047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8325,"log_dups_size":0,"ondisk_log_size":8325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34066432,"num_objects":8317,"num_object_clones":0,"num_object_copies":16634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5395,"num_write_kb":21580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8389","reported_seq":8617,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.016311+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:51:57.016311+0000","last_peered":"2024-09-16T04:51:57.016311+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:51:57.016311+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8389,"log_dups_size":0,"ondisk_log_size":8389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34226176,"num_objects":8356,"num_object_clones":0,"num_object_copies":16712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5140,"num_write_kb":20560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5035}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8389","reported_seq":8519,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.797334+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:51:57.797334+0000","last_peered":"2024-09-16T04:51:57.797334+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:51:57.797334+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8389,"log_dups_size":0,"ondisk_log_size":8389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34328576,"num_objects":8381,"num_object_clones":0,"num_object_copies":16762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5044,"num_write_kb":20176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4916},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8255","reported_seq":8497,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.052975+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:51:57.052975+0000","last_peered":"2024-09-16T04:51:57.052975+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:51:57.052975+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8255,"log_dups_size":0,"ondisk_log_size":8255,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33686738,"num_objects":8226,"num_object_clones":0,"num_object_copies":16452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8227,"num_write_kb":32899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":4930}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8305","reported_seq":15161,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:57.033678+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:51:57.033678+0000","last_peered":"2024-09-16T04:51:57.033678+0000","last_clean":"2024-09-16T04:51:57.033678+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:51:57.033678+0000","last_undegraded":"2024-09-16T04:51:57.033678+0000","last_fullsized":"2024-09-16T04:51:57.033678+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8305,"log_dups_size":0,"ondisk_log_size":8305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33992704,"num_objects":8299,"num_object_clones":0,"num_object_copies":16598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5028,"num_write_kb":20112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8260","reported_seq":8358,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.075703+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:51:57.075703+0000","last_peered":"2024-09-16T04:51:57.075703+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:51:57.075703+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8260,"log_dups_size":0,"ondisk_log_size":8260,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33825385,"num_objects":8259,"num_object_clones":0,"num_object_copies":16518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8259,"num_write_kb":33033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4852}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8473","reported_seq":8575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.058071+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:51:57.058071+0000","last_peered":"2024-09-16T04:51:57.058071+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:51:57.058071+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8473,"log_dups_size":0,"ondisk_log_size":8473,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34674491,"num_objects":8468,"num_object_clones":0,"num_object_copies":16936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8470,"num_write_kb":33864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":4954}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8403","reported_seq":9419,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.847378+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:51:57.847378+0000","last_peered":"2024-09-16T04:51:57.847378+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:51:57.847378+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8403,"log_dups_size":0,"ondisk_log_size":8403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402304,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2881,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5157,"num_write_kb":20628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":1810432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5516},{"shards":"0,2","objects":2881}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8476","reported_seq":8616,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:51:57.808193+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:51:57.808193+0000","last_peered":"2024-09-16T04:51:57.808193+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:51:57.808193+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8476,"log_dups_size":0,"ondisk_log_size":8476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34669161,"num_objects":8465,"num_object_clones":0,"num_object_copies":16930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5245,"num_write_kb":20980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4941},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":546459871,"num_objects":133419,"num_object_clones":0,"num_object_copies":266838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2881,"num_objects_unfound":0,"num_objects_dirty":133419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98653,"num_write_kb":394584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3797,"num_bytes_recovered":15519744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1105563648,"data_stored":1105483631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133575,"ondisk_log_size":133575,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693672,"kb_used_data":414872,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93678168,"statfs":{"total":96636764160,"available":95926444032,"internally_reserved":0,"allocated":424828928,"data_stored":424559945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":285470377},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593757,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451924,"kb_used_data":249476,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93919916,"statfs":{"total":96636764160,"available":96173993984,"internally_reserved":0,"allocated":255463424,"data_stored":255124861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207285934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":690108,"kb_used_data":415532,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93681732,"statfs":{"total":96636764160,"available":95930093568,"internally_reserved":0,"allocated":425504768,"data_stored":425235785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,2,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":425267200,"data_stored":425263721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255688704,"data_stored":255615645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":424607744,"data_stored":424604265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:02.118 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 185 20 135599 135579 2.86162 1.83984 0.0240294 0.0272935 2024-09-16T04:52:03.118 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 186 20 136084 136064 2.85642 1.89453 0.0498525 0.0273426 2024-09-16T04:52:04.118 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 187 20 136573 136553 2.85136 1.91016 0.0256659 0.027391 2024-09-16T04:52:04.714 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:05.118 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 188 20 137055 137035 2.84621 1.88281 0.0210895 0.0274413 2024-09-16T04:52:05.558 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:05.559 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:05.604 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-16T04:52:04.346412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":556175602,"num_objects":135793,"num_object_clones":0,"num_object_copies":271586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2807,"num_objects_unfound":0,"num_objects_dirty":135793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101027,"num_write_kb":404074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3872,"num_bytes_recovered":15822867,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135950,"ondisk_log_size":135950,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1817716,"kb_used_data":1099332,"kb_used_omap":12,"kb_used_meta":718323,"kb_avail":281297804,"statfs":{"total":289910292480,"available":288048951296,"internally_reserved":0,"allocated":1125715968,"data_stored":1124839439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":735562753},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,11,5,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24854528,"num_objects":6068,"num_object_clones":0,"num_object_copies":12136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-243,"num_objects_unfound":0,"num_objects_dirty":6068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6068,"num_write_kb":24272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008858"},"pg_stats":[{"pgid":"2.f","version":"47'8355","reported_seq":8434,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.857603+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:02.857603+0000","last_peered":"2024-09-16T04:52:02.857603+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:02.857603+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8355,"log_dups_size":0,"ondisk_log_size":8355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34222080,"num_objects":8355,"num_object_clones":0,"num_object_copies":16710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8355,"num_write_kb":33420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5014},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8471","reported_seq":8587,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:02.073306+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:02.073306+0000","last_peered":"2024-09-16T04:52:02.073306+0000","last_clean":"2024-09-16T04:52:02.073306+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:02.073306+0000","last_undegraded":"2024-09-16T04:52:02.073306+0000","last_fullsized":"2024-09-16T04:52:02.073306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8471,"log_dups_size":0,"ondisk_log_size":8471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34672640,"num_objects":8465,"num_object_clones":0,"num_object_copies":16930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5440,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8403","reported_seq":8541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.010007+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:02.010007+0000","last_peered":"2024-09-16T04:52:02.010007+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:02.010007+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8403,"log_dups_size":0,"ondisk_log_size":8403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402304,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5142,"num_write_kb":20568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5080}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'8548","reported_seq":8618,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:02.092973+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:02.092973+0000","last_peered":"2024-09-16T04:52:02.092973+0000","last_clean":"2024-09-16T04:52:02.092973+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:02.092973+0000","last_undegraded":"2024-09-16T04:52:02.092973+0000","last_fullsized":"2024-09-16T04:52:02.092973+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8548,"log_dups_size":0,"ondisk_log_size":8548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35012608,"num_objects":8548,"num_object_clones":0,"num_object_copies":17096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8548,"num_write_kb":34192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8508","reported_seq":8727,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:02.948915+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:02.948915+0000","last_peered":"2024-09-16T04:52:02.948915+0000","last_clean":"2024-09-16T04:52:02.948915+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:02.948915+0000","last_undegraded":"2024-09-16T04:52:02.948915+0000","last_fullsized":"2024-09-16T04:52:02.948915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8508,"log_dups_size":0,"ondisk_log_size":8508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34725888,"num_objects":8478,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5540,"num_write_kb":22160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'8592","reported_seq":8725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.869029+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:02.869029+0000","last_peered":"2024-09-16T04:52:02.869029+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:02.869029+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":8592,"log_dups_size":0,"ondisk_log_size":8592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35172352,"num_objects":8587,"num_object_clones":0,"num_object_copies":17174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5585,"num_write_kb":22340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5131},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8451","reported_seq":8599,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.924640+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:02.924640+0000","last_peered":"2024-09-16T04:52:02.924640+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:02.924640+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8451,"log_dups_size":0,"ondisk_log_size":8451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34590720,"num_objects":8445,"num_object_clones":0,"num_object_copies":16890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5106,"num_write_kb":20424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5046},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8477","reported_seq":8693,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:02.792057+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:02.792057+0000","last_peered":"2024-09-16T04:52:02.792057+0000","last_clean":"2024-09-16T04:52:02.792057+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:02.792057+0000","last_undegraded":"2024-09-16T04:52:02.792057+0000","last_fullsized":"2024-09-16T04:52:02.792057+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8477,"log_dups_size":0,"ondisk_log_size":8477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34689024,"num_objects":8469,"num_object_clones":0,"num_object_copies":16938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5547,"num_write_kb":22188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8533","reported_seq":8759,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.002658+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:02.002658+0000","last_peered":"2024-09-16T04:52:02.002658+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:02.002658+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8533,"log_dups_size":0,"ondisk_log_size":8533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34816000,"num_objects":8500,"num_object_clones":0,"num_object_copies":17000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5284,"num_write_kb":21136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5179}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8531","reported_seq":8660,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.909764+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:02.909764+0000","last_peered":"2024-09-16T04:52:02.909764+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:02.909764+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8531,"log_dups_size":0,"ondisk_log_size":8531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34910208,"num_objects":8523,"num_object_clones":0,"num_object_copies":17046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5186,"num_write_kb":20744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5058},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8391","reported_seq":8631,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.095447+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:02.095447+0000","last_peered":"2024-09-16T04:52:02.095447+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:02.095447+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8391,"log_dups_size":0,"ondisk_log_size":8391,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34243794,"num_objects":8362,"num_object_clones":0,"num_object_copies":16724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8363,"num_write_kb":33443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5066}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8464","reported_seq":15320,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:01.994975+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:01.994975+0000","last_peered":"2024-09-16T04:52:01.994975+0000","last_clean":"2024-09-16T04:52:01.994975+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:01.994975+0000","last_undegraded":"2024-09-16T04:52:01.994975+0000","last_fullsized":"2024-09-16T04:52:01.994975+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8464,"log_dups_size":0,"ondisk_log_size":8464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34643968,"num_objects":8458,"num_object_clones":0,"num_object_copies":16916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5187,"num_write_kb":20748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8403","reported_seq":8501,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.055363+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:02.055363+0000","last_peered":"2024-09-16T04:52:02.055363+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:02.055363+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8403,"log_dups_size":0,"ondisk_log_size":8403,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34411113,"num_objects":8402,"num_object_clones":0,"num_object_copies":16804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8402,"num_write_kb":33605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4995}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8631","reported_seq":8734,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.043008+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:02.043008+0000","last_peered":"2024-09-16T04:52:02.043008+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:02.043008+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8631,"log_dups_size":0,"ondisk_log_size":8631,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35321659,"num_objects":8626,"num_object_clones":0,"num_object_copies":17252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":34496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":5112}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8545","reported_seq":9708,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.957072+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:02.957072+0000","last_peered":"2024-09-16T04:52:02.957072+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:02.957072+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8545,"log_dups_size":0,"ondisk_log_size":8545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2807,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5299,"num_write_kb":21196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":2113536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5732},{"shards":"0,2","objects":2807}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8644","reported_seq":8784,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.912921+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:02.912921+0000","last_peered":"2024-09-16T04:52:02.912921+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:02.912921+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8644,"log_dups_size":0,"ondisk_log_size":8644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35357289,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5413,"num_write_kb":21652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5109},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":556175583,"num_objects":135791,"num_object_clones":0,"num_object_copies":271582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2807,"num_objects_unfound":0,"num_objects_dirty":135791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101025,"num_write_kb":404072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15822848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1125703680,"data_stored":1125623663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135947,"ondisk_log_size":135947,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687132,"kb_used_data":420812,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93684708,"statfs":{"total":96636764160,"available":95933140992,"internally_reserved":0,"allocated":430911488,"data_stored":430642505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":272690857},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593758,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447152,"kb_used_data":257184,"kb_used_omap":4,"kb_used_meta":189947,"kb_avail":93924688,"statfs":{"total":96636764160,"available":96178880512,"internally_reserved":0,"allocated":263356416,"data_stored":263017853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":194506414},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,2,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182449,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683432,"kb_used_data":421336,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93688408,"statfs":{"total":96636764160,"available":95936929792,"internally_reserved":0,"allocated":431448064,"data_stored":431179081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":431337472,"data_stored":431333993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":263577600,"data_stored":263504541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":430788608,"data_stored":430785129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:05.606 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:06.118 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 189 20 137497 137477 2.84028 1.72656 0.029566 0.027498 2024-09-16T04:52:06.363 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:06.366 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:06.401 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-16T04:52:06.346904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":556175602,"num_objects":135793,"num_object_clones":0,"num_object_copies":271586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2807,"num_objects_unfound":0,"num_objects_dirty":135793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101027,"num_write_kb":404074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3872,"num_bytes_recovered":15822867,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135950,"ondisk_log_size":135950,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1833504,"kb_used_data":1106800,"kb_used_omap":12,"kb_used_meta":726643,"kb_avail":281282016,"statfs":{"total":289910292480,"available":288032784384,"internally_reserved":0,"allocated":1133363200,"data_stored":1132486671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":744082433},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,3,12,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19877888,"num_objects":4853,"num_object_clones":0,"num_object_copies":9706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-157,"num_objects_unfound":0,"num_objects_dirty":4853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4853,"num_write_kb":19412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":643072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006112"},"pg_stats":[{"pgid":"2.f","version":"47'8355","reported_seq":8434,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.857603+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:02.857603+0000","last_peered":"2024-09-16T04:52:02.857603+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:02.857603+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8355,"log_dups_size":0,"ondisk_log_size":8355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34222080,"num_objects":8355,"num_object_clones":0,"num_object_copies":16710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8355,"num_write_kb":33420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5014},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8471","reported_seq":8587,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:02.073306+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:02.073306+0000","last_peered":"2024-09-16T04:52:02.073306+0000","last_clean":"2024-09-16T04:52:02.073306+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:02.073306+0000","last_undegraded":"2024-09-16T04:52:02.073306+0000","last_fullsized":"2024-09-16T04:52:02.073306+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8471,"log_dups_size":0,"ondisk_log_size":8471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34672640,"num_objects":8465,"num_object_clones":0,"num_object_copies":16930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5440,"num_write_kb":21760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8403","reported_seq":8541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.010007+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:02.010007+0000","last_peered":"2024-09-16T04:52:02.010007+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:02.010007+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8403,"log_dups_size":0,"ondisk_log_size":8403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34402304,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5142,"num_write_kb":20568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5080}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'8548","reported_seq":8618,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:02.092973+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:02.092973+0000","last_peered":"2024-09-16T04:52:02.092973+0000","last_clean":"2024-09-16T04:52:02.092973+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:02.092973+0000","last_undegraded":"2024-09-16T04:52:02.092973+0000","last_fullsized":"2024-09-16T04:52:02.092973+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8548,"log_dups_size":0,"ondisk_log_size":8548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35012608,"num_objects":8548,"num_object_clones":0,"num_object_copies":17096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8548,"num_write_kb":34192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8508","reported_seq":8727,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:02.948915+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:02.948915+0000","last_peered":"2024-09-16T04:52:02.948915+0000","last_clean":"2024-09-16T04:52:02.948915+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:02.948915+0000","last_undegraded":"2024-09-16T04:52:02.948915+0000","last_fullsized":"2024-09-16T04:52:02.948915+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8508,"log_dups_size":0,"ondisk_log_size":8508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34725888,"num_objects":8478,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5540,"num_write_kb":22160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'8592","reported_seq":8725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.869029+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:02.869029+0000","last_peered":"2024-09-16T04:52:02.869029+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:02.869029+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":8592,"log_dups_size":0,"ondisk_log_size":8592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35172352,"num_objects":8587,"num_object_clones":0,"num_object_copies":17174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5585,"num_write_kb":22340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5131},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8451","reported_seq":8599,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.924640+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:02.924640+0000","last_peered":"2024-09-16T04:52:02.924640+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:02.924640+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8451,"log_dups_size":0,"ondisk_log_size":8451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34590720,"num_objects":8445,"num_object_clones":0,"num_object_copies":16890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5106,"num_write_kb":20424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5046},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8477","reported_seq":8693,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:02.792057+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:02.792057+0000","last_peered":"2024-09-16T04:52:02.792057+0000","last_clean":"2024-09-16T04:52:02.792057+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:02.792057+0000","last_undegraded":"2024-09-16T04:52:02.792057+0000","last_fullsized":"2024-09-16T04:52:02.792057+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8477,"log_dups_size":0,"ondisk_log_size":8477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34689024,"num_objects":8469,"num_object_clones":0,"num_object_copies":16938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5547,"num_write_kb":22188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8533","reported_seq":8759,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.002658+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:02.002658+0000","last_peered":"2024-09-16T04:52:02.002658+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:02.002658+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8533,"log_dups_size":0,"ondisk_log_size":8533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34816000,"num_objects":8500,"num_object_clones":0,"num_object_copies":17000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5284,"num_write_kb":21136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5179}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8531","reported_seq":8660,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.909764+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:02.909764+0000","last_peered":"2024-09-16T04:52:02.909764+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:02.909764+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8531,"log_dups_size":0,"ondisk_log_size":8531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34910208,"num_objects":8523,"num_object_clones":0,"num_object_copies":17046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5186,"num_write_kb":20744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5058},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8391","reported_seq":8631,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.095447+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:02.095447+0000","last_peered":"2024-09-16T04:52:02.095447+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:02.095447+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8391,"log_dups_size":0,"ondisk_log_size":8391,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34243794,"num_objects":8362,"num_object_clones":0,"num_object_copies":16724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8363,"num_write_kb":33443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5066}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8464","reported_seq":15320,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:01.994975+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:01.994975+0000","last_peered":"2024-09-16T04:52:01.994975+0000","last_clean":"2024-09-16T04:52:01.994975+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:01.994975+0000","last_undegraded":"2024-09-16T04:52:01.994975+0000","last_fullsized":"2024-09-16T04:52:01.994975+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8464,"log_dups_size":0,"ondisk_log_size":8464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34643968,"num_objects":8458,"num_object_clones":0,"num_object_copies":16916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5187,"num_write_kb":20748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8403","reported_seq":8501,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.055363+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:02.055363+0000","last_peered":"2024-09-16T04:52:02.055363+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:02.055363+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8403,"log_dups_size":0,"ondisk_log_size":8403,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34411113,"num_objects":8402,"num_object_clones":0,"num_object_copies":16804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8402,"num_write_kb":33605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":4995}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8631","reported_seq":8734,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.043008+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:02.043008+0000","last_peered":"2024-09-16T04:52:02.043008+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:02.043008+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8631,"log_dups_size":0,"ondisk_log_size":8631,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35321659,"num_objects":8626,"num_object_clones":0,"num_object_copies":17252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":34496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":5112}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8545","reported_seq":9708,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.957072+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:02.957072+0000","last_peered":"2024-09-16T04:52:02.957072+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:02.957072+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8545,"log_dups_size":0,"ondisk_log_size":8545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2807,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5299,"num_write_kb":21196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":2113536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5732},{"shards":"0,2","objects":2807}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8644","reported_seq":8784,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:02.912921+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:02.912921+0000","last_peered":"2024-09-16T04:52:02.912921+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:02.912921+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8644,"log_dups_size":0,"ondisk_log_size":8644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35357289,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5413,"num_write_kb":21652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5109},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":556175583,"num_objects":135791,"num_object_clones":0,"num_object_copies":271582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2807,"num_objects_unfound":0,"num_objects_dirty":135791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101025,"num_write_kb":404072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3871,"num_bytes_recovered":15822848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1133604864,"data_stored":1133524847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135947,"ondisk_log_size":135947,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687132,"kb_used_data":420812,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93684708,"statfs":{"total":96636764160,"available":95933140992,"internally_reserved":0,"allocated":430911488,"data_stored":430642505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":272690857},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593759,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462940,"kb_used_data":264652,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93908900,"statfs":{"total":96636764160,"available":96162713600,"internally_reserved":0,"allocated":271003648,"data_stored":270665085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":203026094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182449,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683432,"kb_used_data":421336,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93688408,"statfs":{"total":96636764160,"available":95936929792,"internally_reserved":0,"allocated":431448064,"data_stored":431179081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":431337472,"data_stored":431333993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":271478784,"data_stored":271405725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":430788608,"data_stored":430785129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:06.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:52:06.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'8355", 'reported_seq': 8434, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:02.857603+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:52:02.857603+0000', 'last_peered': '2024-09-16T04:52:02.857603+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:52:02.857603+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 8355, 'log_dups_size': 0, 'ondisk_log_size': 8355, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34222080, 'num_objects': 8355, 'num_object_clones': 0, 'num_object_copies': 16710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8355, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8355, 'num_write_kb': 33420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5014}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:06.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:52:06.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'8403", 'reported_seq': 8541, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:02.010007+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:52:02.010007+0000', 'last_peered': '2024-09-16T04:52:02.010007+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:52:02.010007+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 8403, 'log_dups_size': 0, 'ondisk_log_size': 8403, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34402304, 'num_objects': 8399, 'num_object_clones': 0, 'num_object_copies': 16798, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8399, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5142, 'num_write_kb': 20568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 5080}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:06.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:52:06.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'8592", 'reported_seq': 8725, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:02.869029+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:52:02.869029+0000', 'last_peered': '2024-09-16T04:52:02.869029+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:52:02.869029+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 8592, 'log_dups_size': 0, 'ondisk_log_size': 8592, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35172352, 'num_objects': 8587, 'num_object_clones': 0, 'num_object_copies': 17174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8587, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5585, 'num_write_kb': 22340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5131}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:06.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:52:06.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'8451", 'reported_seq': 8599, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:02.924640+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:52:02.924640+0000', 'last_peered': '2024-09-16T04:52:02.924640+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:52:02.924640+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 8451, 'log_dups_size': 0, 'ondisk_log_size': 8451, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34590720, 'num_objects': 8445, 'num_object_clones': 0, 'num_object_copies': 16890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8445, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5106, 'num_write_kb': 20424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5046}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:06.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:52:06.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'8533", 'reported_seq': 8759, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:02.002658+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:52:02.002658+0000', 'last_peered': '2024-09-16T04:52:02.002658+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:52:02.002658+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 8533, 'log_dups_size': 0, 'ondisk_log_size': 8533, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34816000, 'num_objects': 8500, 'num_object_clones': 0, 'num_object_copies': 17000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8500, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5284, 'num_write_kb': 21136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 5179}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:06.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:52:06.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'8531", 'reported_seq': 8660, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:02.909764+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:52:02.909764+0000', 'last_peered': '2024-09-16T04:52:02.909764+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:52:02.909764+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 8531, 'log_dups_size': 0, 'ondisk_log_size': 8531, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34910208, 'num_objects': 8523, 'num_object_clones': 0, 'num_object_copies': 17046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8523, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5186, 'num_write_kb': 20744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5058}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:06.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:52:06.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'8391", 'reported_seq': 8631, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:02.095447+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:52:02.095447+0000', 'last_peered': '2024-09-16T04:52:02.095447+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:52:02.095447+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 8391, 'log_dups_size': 0, 'ondisk_log_size': 8391, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34243794, 'num_objects': 8362, 'num_object_clones': 0, 'num_object_copies': 16724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8362, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8363, 'num_write_kb': 33443, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 5066}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:06.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:52:06.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'8403", 'reported_seq': 8501, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:02.055363+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:52:02.055363+0000', 'last_peered': '2024-09-16T04:52:02.055363+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:52:02.055363+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 8403, 'log_dups_size': 0, 'ondisk_log_size': 8403, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34411113, 'num_objects': 8402, 'num_object_clones': 0, 'num_object_copies': 16804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8402, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8402, 'num_write_kb': 33605, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 4995}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:06.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:52:06.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'8631", 'reported_seq': 8734, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:02.043008+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:52:02.043008+0000', 'last_peered': '2024-09-16T04:52:02.043008+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:52:02.043008+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 8631, 'log_dups_size': 0, 'ondisk_log_size': 8631, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35321659, 'num_objects': 8626, 'num_object_clones': 0, 'num_object_copies': 17252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8628, 'num_write_kb': 34496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 5112}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:06.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:52:06.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'8545", 'reported_seq': 9708, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:02.957072+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:52:02.957072+0000', 'last_peered': '2024-09-16T04:52:02.957072+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:52:02.957072+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 8545, 'log_dups_size': 0, 'ondisk_log_size': 8545, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34983936, 'num_objects': 8541, 'num_object_clones': 0, 'num_object_copies': 17082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2807, 'num_objects_unfound': 0, 'num_objects_dirty': 8541, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5299, 'num_write_kb': 21196, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 516, 'num_bytes_recovered': 2113536, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5732}, {'shards': '0,2', 'objects': 2807}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:06.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:52:06.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'8644", 'reported_seq': 8784, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:02.912921+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:52:02.912921+0000', 'last_peered': '2024-09-16T04:52:02.912921+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:52:02.912921+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 8644, 'log_dups_size': 0, 'ondisk_log_size': 8644, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35357289, 'num_objects': 8633, 'num_object_clones': 0, 'num_object_copies': 17266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8633, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5413, 'num_write_kb': 21652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5109}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:06.405 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:52:07.124 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 190 20 137959 137939 2.83477 1.80469 0.0289489 0.0275519 2024-09-16T04:52:07.536 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:07.536 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/271586 objects degraded (12.542%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":275,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":135793,"data_bytes":556175602,"bytes_used":1877508096,"bytes_avail":288032784384,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":271586,"degraded_ratio":0.12541883602247539,"misplaced_objects":2807,"misplaced_total":271586,"misplaced_ratio":0.010335584308469507,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":53562,"recovering_keys_per_sec":0,"num_objects_recovered":157,"num_bytes_recovered":643072,"num_keys_recovered":0,"write_bytes_sec":1655647,"read_op_per_sec":0,"write_op_per_sec":404},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:52:07.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:52:07.603 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:08.123 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 191 20 138421 138401 2.82938 1.80469 0.0705356 0.027604 2024-09-16T04:52:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:08.467 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:08.542 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-16T04:52:08.347371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":565895410,"num_objects":138166,"num_object_clones":0,"num_object_copies":276332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2712,"num_objects_unfound":0,"num_objects_dirty":138166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103400,"num_write_kb":413566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3967,"num_bytes_recovered":16211987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138323,"ondisk_log_size":138323,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1853604,"kb_used_data":1118580,"kb_used_omap":12,"kb_used_meta":734963,"kb_avail":281261916,"statfs":{"total":289910292480,"available":288012201984,"internally_reserved":0,"allocated":1145425920,"data_stored":1144549391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":752602113},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,1,1,6,3,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29597696,"num_objects":7226,"num_object_clones":0,"num_object_copies":14452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":7226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7226,"num_write_kb":28904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004682"},"pg_stats":[{"pgid":"2.f","version":"47'8510","reported_seq":8588,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.985993+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:07.985993+0000","last_peered":"2024-09-16T04:52:07.985993+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:07.985993+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8510,"log_dups_size":0,"ondisk_log_size":8510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34856960,"num_objects":8510,"num_object_clones":0,"num_object_copies":17020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8510,"num_write_kb":34040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5169},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8619","reported_seq":8735,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:07.123731+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:07.123731+0000","last_peered":"2024-09-16T04:52:07.123731+0000","last_clean":"2024-09-16T04:52:07.123731+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:07.123731+0000","last_undegraded":"2024-09-16T04:52:07.123731+0000","last_fullsized":"2024-09-16T04:52:07.123731+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8619,"log_dups_size":0,"ondisk_log_size":8619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35278848,"num_objects":8613,"num_object_clones":0,"num_object_copies":17226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5588,"num_write_kb":22352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8545","reported_seq":8683,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:06.999355+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:06.999355+0000","last_peered":"2024-09-16T04:52:06.999355+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:06.999355+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8545,"log_dups_size":0,"ondisk_log_size":8545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5284,"num_write_kb":21136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5222}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'8694","reported_seq":8764,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:07.120868+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:07.120868+0000","last_peered":"2024-09-16T04:52:07.120868+0000","last_clean":"2024-09-16T04:52:07.120868+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:07.120868+0000","last_undegraded":"2024-09-16T04:52:07.120868+0000","last_fullsized":"2024-09-16T04:52:07.120868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8694,"log_dups_size":0,"ondisk_log_size":8694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35610624,"num_objects":8694,"num_object_clones":0,"num_object_copies":17388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8694,"num_write_kb":34776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8660","reported_seq":8879,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:07.932461+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:07.932461+0000","last_peered":"2024-09-16T04:52:07.932461+0000","last_clean":"2024-09-16T04:52:07.932461+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:07.932461+0000","last_undegraded":"2024-09-16T04:52:07.932461+0000","last_fullsized":"2024-09-16T04:52:07.932461+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8660,"log_dups_size":0,"ondisk_log_size":8660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35348480,"num_objects":8630,"num_object_clones":0,"num_object_copies":17260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5692,"num_write_kb":22768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'8728","reported_seq":8861,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.907615+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:07.907615+0000","last_peered":"2024-09-16T04:52:07.907615+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:07.907615+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":8728,"log_dups_size":0,"ondisk_log_size":8728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35729408,"num_objects":8723,"num_object_clones":0,"num_object_copies":17446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5721,"num_write_kb":22884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5267},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8593","reported_seq":8739,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.779861+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:07.779861+0000","last_peered":"2024-09-16T04:52:07.779861+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:07.779861+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8593,"log_dups_size":0,"ondisk_log_size":8593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35172352,"num_objects":8587,"num_object_clones":0,"num_object_copies":17174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5248,"num_write_kb":20992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5188},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8617","reported_seq":8833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:07.841548+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:07.841548+0000","last_peered":"2024-09-16T04:52:07.841548+0000","last_clean":"2024-09-16T04:52:07.841548+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:07.841548+0000","last_undegraded":"2024-09-16T04:52:07.841548+0000","last_fullsized":"2024-09-16T04:52:07.841548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8617,"log_dups_size":0,"ondisk_log_size":8617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35262464,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5687,"num_write_kb":22748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8695","reported_seq":8923,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.065196+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:07.065196+0000","last_peered":"2024-09-16T04:52:07.065196+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:07.065196+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8695,"log_dups_size":0,"ondisk_log_size":8695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35479552,"num_objects":8662,"num_object_clones":0,"num_object_copies":17324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5446,"num_write_kb":21784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5341}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8670","reported_seq":8800,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.908937+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:07.908937+0000","last_peered":"2024-09-16T04:52:07.908937+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:07.908937+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8670,"log_dups_size":0,"ondisk_log_size":8670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35479552,"num_objects":8662,"num_object_clones":0,"num_object_copies":17324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5325,"num_write_kb":21300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5197},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8528","reported_seq":8770,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:06.962438+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:06.962438+0000","last_peered":"2024-09-16T04:52:06.962438+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:06.962438+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8528,"log_dups_size":0,"ondisk_log_size":8528,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34804946,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8500,"num_write_kb":33991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5203}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8606","reported_seq":15462,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:07.105709+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:07.105709+0000","last_peered":"2024-09-16T04:52:07.105709+0000","last_clean":"2024-09-16T04:52:07.105709+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:07.105709+0000","last_undegraded":"2024-09-16T04:52:07.105709+0000","last_fullsized":"2024-09-16T04:52:07.105709+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8606,"log_dups_size":0,"ondisk_log_size":8606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35225600,"num_objects":8600,"num_object_clones":0,"num_object_copies":17200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5329,"num_write_kb":21316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8548","reported_seq":8645,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.141505+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:07.141505+0000","last_peered":"2024-09-16T04:52:07.141505+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:07.141505+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8548,"log_dups_size":0,"ondisk_log_size":8548,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35005033,"num_objects":8547,"num_object_clones":0,"num_object_copies":17094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8547,"num_write_kb":34185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5140}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8817","reported_seq":8920,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.102720+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:07.102720+0000","last_peered":"2024-09-16T04:52:07.102720+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:07.102720+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8817,"log_dups_size":0,"ondisk_log_size":8817,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36083515,"num_objects":8812,"num_object_clones":0,"num_object_copies":17624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8814,"num_write_kb":35240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":5298}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8683","reported_seq":10036,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:08.002053+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:08.002053+0000","last_peered":"2024-09-16T04:52:08.002053+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:08.002053+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8683,"log_dups_size":0,"ondisk_log_size":8683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35549184,"num_objects":8679,"num_object_clones":0,"num_object_copies":17358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2712,"num_objects_unfound":0,"num_objects_dirty":8679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5437,"num_write_kb":21748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":611,"num_bytes_recovered":2502656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5965},{"shards":"0,2","objects":2712}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8807","reported_seq":8946,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.888978+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:07.888978+0000","last_peered":"2024-09-16T04:52:07.888978+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:07.888978+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8807,"log_dups_size":0,"ondisk_log_size":8807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36024937,"num_objects":8796,"num_object_clones":0,"num_object_copies":17592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5576,"num_write_kb":22304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5272},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":565895391,"num_objects":138164,"num_object_clones":0,"num_object_copies":276328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2712,"num_objects_unfound":0,"num_objects_dirty":138164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103398,"num_write_kb":413564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3966,"num_bytes_recovered":16211968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1145643008,"data_stored":1145562991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138320,"ondisk_log_size":138320,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697220,"kb_used_data":426740,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93674620,"statfs":{"total":96636764160,"available":95922810880,"internally_reserved":0,"allocated":436981760,"data_stored":436712777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":276950697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593759,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462940,"kb_used_data":264652,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93908900,"statfs":{"total":96636764160,"available":96162713600,"internally_reserved":0,"allocated":271003648,"data_stored":270665085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":203026094},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693444,"kb_used_data":427188,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93678396,"statfs":{"total":96636764160,"available":95926677504,"internally_reserved":0,"allocated":437440512,"data_stored":437171529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,0,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":437268480,"data_stored":437265001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":271478784,"data_stored":271405725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":436895744,"data_stored":436892265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:09.123 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 192 20 138901 138881 2.82441 1.875 0.0160657 0.0276529 2024-09-16T04:52:10.123 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 193 20 139383 139363 2.81953 1.88281 0.0123495 0.0277008 2024-09-16T04:52:11.123 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 194 20 139878 139858 2.81496 1.93359 0.0186674 0.0277461 2024-09-16T04:52:11.545 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:12.123 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 195 20 140350 140330 2.80998 1.84375 0.0126059 0.0277944 2024-09-16T04:52:12.439 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:12.441 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:12.500 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-16T04:52:12.348335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":570847474,"num_objects":139375,"num_object_clones":0,"num_object_copies":278750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2712,"num_objects_unfound":0,"num_objects_dirty":139375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104609,"num_write_kb":418402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3967,"num_bytes_recovered":16211987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139532,"ondisk_log_size":139532,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1875332,"kb_used_data":1131988,"kb_used_omap":12,"kb_used_meta":743283,"kb_avail":281240188,"statfs":{"total":289910292480,"available":287989952512,"internally_reserved":0,"allocated":1159155712,"data_stored":1158279183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":761121793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,4,2,5,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24387584,"num_objects":5954,"num_object_clones":0,"num_object_copies":11908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-169,"num_objects_unfound":0,"num_objects_dirty":5954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5954,"num_write_kb":23816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002860"},"pg_stats":[{"pgid":"2.f","version":"47'8510","reported_seq":8588,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.985993+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:07.985993+0000","last_peered":"2024-09-16T04:52:07.985993+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:07.985993+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8510,"log_dups_size":0,"ondisk_log_size":8510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34856960,"num_objects":8510,"num_object_clones":0,"num_object_copies":17020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8510,"num_write_kb":34040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5169},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8762","reported_seq":8878,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:12.044365+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:12.044365+0000","last_peered":"2024-09-16T04:52:12.044365+0000","last_clean":"2024-09-16T04:52:12.044365+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:12.044365+0000","last_undegraded":"2024-09-16T04:52:12.044365+0000","last_fullsized":"2024-09-16T04:52:12.044365+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8762,"log_dups_size":0,"ondisk_log_size":8762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35864576,"num_objects":8756,"num_object_clones":0,"num_object_copies":17512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5731,"num_write_kb":22924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8690","reported_seq":8828,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.153017+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:12.153017+0000","last_peered":"2024-09-16T04:52:12.153017+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:12.153017+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8690,"log_dups_size":0,"ondisk_log_size":8690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35577856,"num_objects":8686,"num_object_clones":0,"num_object_copies":17372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5429,"num_write_kb":21716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5367}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'8854","reported_seq":8924,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:12.150625+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:12.150625+0000","last_peered":"2024-09-16T04:52:12.150625+0000","last_clean":"2024-09-16T04:52:12.150625+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:12.150625+0000","last_undegraded":"2024-09-16T04:52:12.150625+0000","last_fullsized":"2024-09-16T04:52:12.150625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8854,"log_dups_size":0,"ondisk_log_size":8854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36265984,"num_objects":8854,"num_object_clones":0,"num_object_copies":17708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8854,"num_write_kb":35416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8660","reported_seq":8879,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:07.932461+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:07.932461+0000","last_peered":"2024-09-16T04:52:07.932461+0000","last_clean":"2024-09-16T04:52:07.932461+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:07.932461+0000","last_undegraded":"2024-09-16T04:52:07.932461+0000","last_fullsized":"2024-09-16T04:52:07.932461+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8660,"log_dups_size":0,"ondisk_log_size":8660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35348480,"num_objects":8630,"num_object_clones":0,"num_object_copies":17260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5692,"num_write_kb":22768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'8728","reported_seq":8861,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.907615+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:07.907615+0000","last_peered":"2024-09-16T04:52:07.907615+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:07.907615+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":8728,"log_dups_size":0,"ondisk_log_size":8728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35729408,"num_objects":8723,"num_object_clones":0,"num_object_copies":17446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5721,"num_write_kb":22884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5267},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8593","reported_seq":8739,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.779861+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:07.779861+0000","last_peered":"2024-09-16T04:52:07.779861+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:07.779861+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8593,"log_dups_size":0,"ondisk_log_size":8593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35172352,"num_objects":8587,"num_object_clones":0,"num_object_copies":17174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5248,"num_write_kb":20992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5188},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8617","reported_seq":8833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:07.841548+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:07.841548+0000","last_peered":"2024-09-16T04:52:07.841548+0000","last_clean":"2024-09-16T04:52:07.841548+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:07.841548+0000","last_undegraded":"2024-09-16T04:52:07.841548+0000","last_fullsized":"2024-09-16T04:52:07.841548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8617,"log_dups_size":0,"ondisk_log_size":8617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35262464,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5687,"num_write_kb":22748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8842","reported_seq":9070,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.108951+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:12.108951+0000","last_peered":"2024-09-16T04:52:12.108951+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:12.108951+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8842,"log_dups_size":0,"ondisk_log_size":8842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36081664,"num_objects":8809,"num_object_clones":0,"num_object_copies":17618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5593,"num_write_kb":22372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5488}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8670","reported_seq":8800,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.908937+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:07.908937+0000","last_peered":"2024-09-16T04:52:07.908937+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:07.908937+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8670,"log_dups_size":0,"ondisk_log_size":8670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35479552,"num_objects":8662,"num_object_clones":0,"num_object_copies":17324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5325,"num_write_kb":21300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5197},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8681","reported_seq":8923,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.148033+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:12.148033+0000","last_peered":"2024-09-16T04:52:12.148033+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:12.148033+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8681,"log_dups_size":0,"ondisk_log_size":8681,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35431634,"num_objects":8652,"num_object_clones":0,"num_object_copies":17304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8653,"num_write_kb":34603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5356}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8763","reported_seq":15619,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:12.134565+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:12.134565+0000","last_peered":"2024-09-16T04:52:12.134565+0000","last_clean":"2024-09-16T04:52:12.134565+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:12.134565+0000","last_undegraded":"2024-09-16T04:52:12.134565+0000","last_fullsized":"2024-09-16T04:52:12.134565+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8763,"log_dups_size":0,"ondisk_log_size":8763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35868672,"num_objects":8757,"num_object_clones":0,"num_object_copies":17514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5486,"num_write_kb":21944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8699","reported_seq":8797,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.125259+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:12.125259+0000","last_peered":"2024-09-16T04:52:12.125259+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:12.125259+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8699,"log_dups_size":0,"ondisk_log_size":8699,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35623529,"num_objects":8698,"num_object_clones":0,"num_object_copies":17396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8698,"num_write_kb":34789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8970","reported_seq":9073,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.153041+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:12.153041+0000","last_peered":"2024-09-16T04:52:12.153041+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:12.153041+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8970,"log_dups_size":0,"ondisk_log_size":8970,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36710203,"num_objects":8965,"num_object_clones":0,"num_object_copies":17930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8967,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":5451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8683","reported_seq":10036,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:08.002053+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:08.002053+0000","last_peered":"2024-09-16T04:52:08.002053+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:08.002053+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8683,"log_dups_size":0,"ondisk_log_size":8683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35549184,"num_objects":8679,"num_object_clones":0,"num_object_copies":17358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2712,"num_objects_unfound":0,"num_objects_dirty":8679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5437,"num_write_kb":21748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":611,"num_bytes_recovered":2502656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5965},{"shards":"0,2","objects":2712}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8807","reported_seq":8946,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.888978+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:07.888978+0000","last_peered":"2024-09-16T04:52:07.888978+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:07.888978+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8807,"log_dups_size":0,"ondisk_log_size":8807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36024937,"num_objects":8796,"num_object_clones":0,"num_object_copies":17592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5576,"num_write_kb":22304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5272},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":570847455,"num_objects":139373,"num_object_clones":0,"num_object_copies":278746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2712,"num_objects_unfound":0,"num_objects_dirty":139373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104607,"num_write_kb":418400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3966,"num_bytes_recovered":16211968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1159573504,"data_stored":1159493487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139529,"ondisk_log_size":139529,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707460,"kb_used_data":432820,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93664380,"statfs":{"total":96636764160,"available":95912325120,"internally_reserved":0,"allocated":443207680,"data_stored":442938697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593760,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474428,"kb_used_data":271980,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93897412,"statfs":{"total":96636764160,"available":96150949888,"internally_reserved":0,"allocated":278507520,"data_stored":278168957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207285934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693444,"kb_used_data":427188,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93678396,"statfs":{"total":96636764160,"available":95926677504,"internally_reserved":0,"allocated":437440512,"data_stored":437171529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,0,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":437268480,"data_stored":437265001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279207936,"data_stored":279134877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":443097088,"data_stored":443093609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:12.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:13.124 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 196 20 140822 140802 2.80505 1.84375 0.0422022 0.0278443 2024-09-16T04:52:13.393 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:13.395 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:13.453 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-16T04:52:12.348335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":570847474,"num_objects":139375,"num_object_clones":0,"num_object_copies":278750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2712,"num_objects_unfound":0,"num_objects_dirty":139375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104609,"num_write_kb":418402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3967,"num_bytes_recovered":16211987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139532,"ondisk_log_size":139532,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1875332,"kb_used_data":1131988,"kb_used_omap":12,"kb_used_meta":743283,"kb_avail":281240188,"statfs":{"total":289910292480,"available":287989952512,"internally_reserved":0,"allocated":1159155712,"data_stored":1158279183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":761121793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,4,2,5,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24387584,"num_objects":5954,"num_object_clones":0,"num_object_copies":11908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-169,"num_objects_unfound":0,"num_objects_dirty":5954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5954,"num_write_kb":23816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002860"},"pg_stats":[{"pgid":"2.f","version":"47'8510","reported_seq":8588,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.985993+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:07.985993+0000","last_peered":"2024-09-16T04:52:07.985993+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:07.985993+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8510,"log_dups_size":0,"ondisk_log_size":8510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34856960,"num_objects":8510,"num_object_clones":0,"num_object_copies":17020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8510,"num_write_kb":34040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5169},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8762","reported_seq":8878,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:12.044365+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:12.044365+0000","last_peered":"2024-09-16T04:52:12.044365+0000","last_clean":"2024-09-16T04:52:12.044365+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:12.044365+0000","last_undegraded":"2024-09-16T04:52:12.044365+0000","last_fullsized":"2024-09-16T04:52:12.044365+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8762,"log_dups_size":0,"ondisk_log_size":8762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35864576,"num_objects":8756,"num_object_clones":0,"num_object_copies":17512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5731,"num_write_kb":22924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8690","reported_seq":8828,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.153017+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:12.153017+0000","last_peered":"2024-09-16T04:52:12.153017+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:12.153017+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8690,"log_dups_size":0,"ondisk_log_size":8690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35577856,"num_objects":8686,"num_object_clones":0,"num_object_copies":17372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5429,"num_write_kb":21716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5367}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'8854","reported_seq":8924,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:12.150625+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:12.150625+0000","last_peered":"2024-09-16T04:52:12.150625+0000","last_clean":"2024-09-16T04:52:12.150625+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:12.150625+0000","last_undegraded":"2024-09-16T04:52:12.150625+0000","last_fullsized":"2024-09-16T04:52:12.150625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8854,"log_dups_size":0,"ondisk_log_size":8854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36265984,"num_objects":8854,"num_object_clones":0,"num_object_copies":17708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8854,"num_write_kb":35416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8660","reported_seq":8879,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:07.932461+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:07.932461+0000","last_peered":"2024-09-16T04:52:07.932461+0000","last_clean":"2024-09-16T04:52:07.932461+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:07.932461+0000","last_undegraded":"2024-09-16T04:52:07.932461+0000","last_fullsized":"2024-09-16T04:52:07.932461+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8660,"log_dups_size":0,"ondisk_log_size":8660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35348480,"num_objects":8630,"num_object_clones":0,"num_object_copies":17260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5692,"num_write_kb":22768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'8728","reported_seq":8861,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.907615+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:07.907615+0000","last_peered":"2024-09-16T04:52:07.907615+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:07.907615+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":8728,"log_dups_size":0,"ondisk_log_size":8728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35729408,"num_objects":8723,"num_object_clones":0,"num_object_copies":17446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5721,"num_write_kb":22884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5267},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8593","reported_seq":8739,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.779861+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:07.779861+0000","last_peered":"2024-09-16T04:52:07.779861+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:07.779861+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8593,"log_dups_size":0,"ondisk_log_size":8593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35172352,"num_objects":8587,"num_object_clones":0,"num_object_copies":17174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5248,"num_write_kb":20992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5188},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8617","reported_seq":8833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:07.841548+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:07.841548+0000","last_peered":"2024-09-16T04:52:07.841548+0000","last_clean":"2024-09-16T04:52:07.841548+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:07.841548+0000","last_undegraded":"2024-09-16T04:52:07.841548+0000","last_fullsized":"2024-09-16T04:52:07.841548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8617,"log_dups_size":0,"ondisk_log_size":8617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35262464,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5687,"num_write_kb":22748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8842","reported_seq":9070,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.108951+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:12.108951+0000","last_peered":"2024-09-16T04:52:12.108951+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:12.108951+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8842,"log_dups_size":0,"ondisk_log_size":8842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36081664,"num_objects":8809,"num_object_clones":0,"num_object_copies":17618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5593,"num_write_kb":22372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5488}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8670","reported_seq":8800,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.908937+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:07.908937+0000","last_peered":"2024-09-16T04:52:07.908937+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:07.908937+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8670,"log_dups_size":0,"ondisk_log_size":8670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35479552,"num_objects":8662,"num_object_clones":0,"num_object_copies":17324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5325,"num_write_kb":21300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5197},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8681","reported_seq":8923,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.148033+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:12.148033+0000","last_peered":"2024-09-16T04:52:12.148033+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:12.148033+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8681,"log_dups_size":0,"ondisk_log_size":8681,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35431634,"num_objects":8652,"num_object_clones":0,"num_object_copies":17304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8653,"num_write_kb":34603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5356}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8763","reported_seq":15619,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:12.134565+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:12.134565+0000","last_peered":"2024-09-16T04:52:12.134565+0000","last_clean":"2024-09-16T04:52:12.134565+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:12.134565+0000","last_undegraded":"2024-09-16T04:52:12.134565+0000","last_fullsized":"2024-09-16T04:52:12.134565+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8763,"log_dups_size":0,"ondisk_log_size":8763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35868672,"num_objects":8757,"num_object_clones":0,"num_object_copies":17514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5486,"num_write_kb":21944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8699","reported_seq":8797,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.125259+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:12.125259+0000","last_peered":"2024-09-16T04:52:12.125259+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:12.125259+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8699,"log_dups_size":0,"ondisk_log_size":8699,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35623529,"num_objects":8698,"num_object_clones":0,"num_object_copies":17396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8698,"num_write_kb":34789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8970","reported_seq":9073,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.153041+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:12.153041+0000","last_peered":"2024-09-16T04:52:12.153041+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:12.153041+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8970,"log_dups_size":0,"ondisk_log_size":8970,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36710203,"num_objects":8965,"num_object_clones":0,"num_object_copies":17930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8967,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":5451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8683","reported_seq":10036,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:08.002053+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:08.002053+0000","last_peered":"2024-09-16T04:52:08.002053+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:08.002053+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8683,"log_dups_size":0,"ondisk_log_size":8683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35549184,"num_objects":8679,"num_object_clones":0,"num_object_copies":17358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2712,"num_objects_unfound":0,"num_objects_dirty":8679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5437,"num_write_kb":21748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":611,"num_bytes_recovered":2502656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5965},{"shards":"0,2","objects":2712}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8807","reported_seq":8946,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:07.888978+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:07.888978+0000","last_peered":"2024-09-16T04:52:07.888978+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:07.888978+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8807,"log_dups_size":0,"ondisk_log_size":8807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36024937,"num_objects":8796,"num_object_clones":0,"num_object_copies":17592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5576,"num_write_kb":22304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5272},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":570847455,"num_objects":139373,"num_object_clones":0,"num_object_copies":278746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2712,"num_objects_unfound":0,"num_objects_dirty":139373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104607,"num_write_kb":418400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3966,"num_bytes_recovered":16211968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1159573504,"data_stored":1159493487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139529,"ondisk_log_size":139529,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707460,"kb_used_data":432820,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93664380,"statfs":{"total":96636764160,"available":95912325120,"internally_reserved":0,"allocated":443207680,"data_stored":442938697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593760,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474428,"kb_used_data":271980,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93897412,"statfs":{"total":96636764160,"available":96150949888,"internally_reserved":0,"allocated":278507520,"data_stored":278168957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207285934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693444,"kb_used_data":427188,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93678396,"statfs":{"total":96636764160,"available":95926677504,"internally_reserved":0,"allocated":437440512,"data_stored":437171529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,0,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":437268480,"data_stored":437265001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279207936,"data_stored":279134877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":443097088,"data_stored":443093609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:13.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:52:13.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'8510", 'reported_seq': 8588, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:07.985993+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:52:07.985993+0000', 'last_peered': '2024-09-16T04:52:07.985993+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:52:07.985993+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 8510, 'log_dups_size': 0, 'ondisk_log_size': 8510, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34856960, 'num_objects': 8510, 'num_object_clones': 0, 'num_object_copies': 17020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8510, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8510, 'num_write_kb': 34040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5169}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:13.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:52:13.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'8690", 'reported_seq': 8828, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:12.153017+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:52:12.153017+0000', 'last_peered': '2024-09-16T04:52:12.153017+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:52:12.153017+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 8690, 'log_dups_size': 0, 'ondisk_log_size': 8690, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35577856, 'num_objects': 8686, 'num_object_clones': 0, 'num_object_copies': 17372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8686, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5429, 'num_write_kb': 21716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 5367}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:13.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:52:13.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'8728", 'reported_seq': 8861, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:07.907615+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:52:07.907615+0000', 'last_peered': '2024-09-16T04:52:07.907615+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:52:07.907615+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 8728, 'log_dups_size': 0, 'ondisk_log_size': 8728, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35729408, 'num_objects': 8723, 'num_object_clones': 0, 'num_object_copies': 17446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8723, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5721, 'num_write_kb': 22884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5267}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:13.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:52:13.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'8593", 'reported_seq': 8739, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:07.779861+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:52:07.779861+0000', 'last_peered': '2024-09-16T04:52:07.779861+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:52:07.779861+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 8593, 'log_dups_size': 0, 'ondisk_log_size': 8593, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35172352, 'num_objects': 8587, 'num_object_clones': 0, 'num_object_copies': 17174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8587, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5248, 'num_write_kb': 20992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5188}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:13.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:52:13.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'8842", 'reported_seq': 9070, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:12.108951+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:52:12.108951+0000', 'last_peered': '2024-09-16T04:52:12.108951+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:52:12.108951+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 8842, 'log_dups_size': 0, 'ondisk_log_size': 8842, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36081664, 'num_objects': 8809, 'num_object_clones': 0, 'num_object_copies': 17618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8809, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5593, 'num_write_kb': 22372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 5488}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:13.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:52:13.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'8670", 'reported_seq': 8800, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:07.908937+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:52:07.908937+0000', 'last_peered': '2024-09-16T04:52:07.908937+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:52:07.908937+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 8670, 'log_dups_size': 0, 'ondisk_log_size': 8670, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35479552, 'num_objects': 8662, 'num_object_clones': 0, 'num_object_copies': 17324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8662, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5325, 'num_write_kb': 21300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5197}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:13.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:52:13.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'8681", 'reported_seq': 8923, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:12.148033+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:52:12.148033+0000', 'last_peered': '2024-09-16T04:52:12.148033+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:52:12.148033+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 8681, 'log_dups_size': 0, 'ondisk_log_size': 8681, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35431634, 'num_objects': 8652, 'num_object_clones': 0, 'num_object_copies': 17304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8652, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8653, 'num_write_kb': 34603, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 5356}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:13.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:52:13.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'8699", 'reported_seq': 8797, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:12.125259+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:52:12.125259+0000', 'last_peered': '2024-09-16T04:52:12.125259+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:52:12.125259+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 8699, 'log_dups_size': 0, 'ondisk_log_size': 8699, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35623529, 'num_objects': 8698, 'num_object_clones': 0, 'num_object_copies': 17396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8698, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8698, 'num_write_kb': 34789, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 5291}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:13.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:52:13.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'8970", 'reported_seq': 9073, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:12.153041+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:52:12.153041+0000', 'last_peered': '2024-09-16T04:52:12.153041+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:52:12.153041+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 8970, 'log_dups_size': 0, 'ondisk_log_size': 8970, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36710203, 'num_objects': 8965, 'num_object_clones': 0, 'num_object_copies': 17930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8965, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8967, 'num_write_kb': 35852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 5451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:13.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:52:13.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'8683", 'reported_seq': 10036, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:08.002053+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:52:08.002053+0000', 'last_peered': '2024-09-16T04:52:08.002053+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:52:08.002053+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 8683, 'log_dups_size': 0, 'ondisk_log_size': 8683, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35549184, 'num_objects': 8679, 'num_object_clones': 0, 'num_object_copies': 17358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2712, 'num_objects_unfound': 0, 'num_objects_dirty': 8679, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5437, 'num_write_kb': 21748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 611, 'num_bytes_recovered': 2502656, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5965}, {'shards': '0,2', 'objects': 2712}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:13.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:52:13.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'8807", 'reported_seq': 8946, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:07.888978+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:52:07.888978+0000', 'last_peered': '2024-09-16T04:52:07.888978+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:52:07.888978+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 8807, 'log_dups_size': 0, 'ondisk_log_size': 8807, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36024937, 'num_objects': 8796, 'num_object_clones': 0, 'num_object_copies': 17592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8796, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5576, 'num_write_kb': 22304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5272}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:13.457 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:52:14.124 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 197 20 141279 141259 2.79987 1.78516 0.0371148 0.0278947 2024-09-16T04:52:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:14.468 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/278750 objects degraded (12.220%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":282,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":139375,"data_bytes":570847474,"bytes_used":1920339968,"bytes_avail":287989952512,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":278750,"degraded_ratio":0.12219551569506726,"misplaced_objects":2712,"misplaced_total":278750,"misplaced_ratio":0.0097291479820627805,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":57671,"recovering_keys_per_sec":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"write_bytes_sec":2031814,"read_op_per_sec":0,"write_op_per_sec":496},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:52:14.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:52:14.509 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:15.124 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 198 20 141742 141722 2.79487 1.80859 0.0269164 0.0279456 2024-09-16T04:52:15.385 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:15.439 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-16T04:52:14.348790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":575758578,"num_objects":140574,"num_object_clones":0,"num_object_copies":281148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2619,"num_objects_unfound":0,"num_objects_dirty":140574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105808,"num_write_kb":423198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4060,"num_bytes_recovered":16592915,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140731,"ondisk_log_size":140731,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1889644,"kb_used_data":1137980,"kb_used_omap":12,"kb_used_meta":751603,"kb_avail":281225876,"statfs":{"total":289910292480,"available":287975297024,"internally_reserved":0,"allocated":1165291520,"data_stored":1164414991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":769641473},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,2,5,10,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24363008,"num_objects":5948,"num_object_clones":0,"num_object_copies":11896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-262,"num_objects_unfound":0,"num_objects_dirty":5948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5948,"num_write_kb":23792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":1073152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002875"},"pg_stats":[{"pgid":"2.f","version":"47'8647","reported_seq":8726,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.929264+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:12.929264+0000","last_peered":"2024-09-16T04:52:12.929264+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:12.929264+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8647,"log_dups_size":0,"ondisk_log_size":8647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35418112,"num_objects":8647,"num_object_clones":0,"num_object_copies":17294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8647,"num_write_kb":34588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5306},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8762","reported_seq":8878,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:12.044365+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:12.044365+0000","last_peered":"2024-09-16T04:52:12.044365+0000","last_clean":"2024-09-16T04:52:12.044365+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:12.044365+0000","last_undegraded":"2024-09-16T04:52:12.044365+0000","last_fullsized":"2024-09-16T04:52:12.044365+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8762,"log_dups_size":0,"ondisk_log_size":8762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35864576,"num_objects":8756,"num_object_clones":0,"num_object_copies":17512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5731,"num_write_kb":22924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8690","reported_seq":8828,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.153017+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:12.153017+0000","last_peered":"2024-09-16T04:52:12.153017+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:12.153017+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8690,"log_dups_size":0,"ondisk_log_size":8690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35577856,"num_objects":8686,"num_object_clones":0,"num_object_copies":17372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5429,"num_write_kb":21716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5367}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'8854","reported_seq":8924,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:12.150625+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:12.150625+0000","last_peered":"2024-09-16T04:52:12.150625+0000","last_clean":"2024-09-16T04:52:12.150625+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:12.150625+0000","last_undegraded":"2024-09-16T04:52:12.150625+0000","last_fullsized":"2024-09-16T04:52:12.150625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8854,"log_dups_size":0,"ondisk_log_size":8854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36265984,"num_objects":8854,"num_object_clones":0,"num_object_copies":17708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8854,"num_write_kb":35416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8822","reported_seq":9041,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:12.980018+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:12.980018+0000","last_peered":"2024-09-16T04:52:12.980018+0000","last_clean":"2024-09-16T04:52:12.980018+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:12.980018+0000","last_undegraded":"2024-09-16T04:52:12.980018+0000","last_fullsized":"2024-09-16T04:52:12.980018+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8822,"log_dups_size":0,"ondisk_log_size":8822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36012032,"num_objects":8792,"num_object_clones":0,"num_object_copies":17584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5854,"num_write_kb":23416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'8889","reported_seq":9020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:13.003073+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:13.003073+0000","last_peered":"2024-09-16T04:52:13.003073+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:13.003073+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":8889,"log_dups_size":0,"ondisk_log_size":8889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36388864,"num_objects":8884,"num_object_clones":0,"num_object_copies":17768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5882,"num_write_kb":23528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5428},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8738","reported_seq":8886,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.894502+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:12.894502+0000","last_peered":"2024-09-16T04:52:12.894502+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:12.894502+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8738,"log_dups_size":0,"ondisk_log_size":8738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35766272,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5393,"num_write_kb":21572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5333},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8763","reported_seq":8979,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:12.988953+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:12.988953+0000","last_peered":"2024-09-16T04:52:12.988953+0000","last_clean":"2024-09-16T04:52:12.988953+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:12.988953+0000","last_undegraded":"2024-09-16T04:52:12.988953+0000","last_fullsized":"2024-09-16T04:52:12.988953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8763,"log_dups_size":0,"ondisk_log_size":8763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35860480,"num_objects":8755,"num_object_clones":0,"num_object_copies":17510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5833,"num_write_kb":23332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8842","reported_seq":9070,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.108951+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:12.108951+0000","last_peered":"2024-09-16T04:52:12.108951+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:12.108951+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8842,"log_dups_size":0,"ondisk_log_size":8842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36081664,"num_objects":8809,"num_object_clones":0,"num_object_copies":17618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5593,"num_write_kb":22372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5488}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8833","reported_seq":8963,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.922483+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:12.922483+0000","last_peered":"2024-09-16T04:52:12.922483+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:12.922483+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8833,"log_dups_size":0,"ondisk_log_size":8833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36147200,"num_objects":8825,"num_object_clones":0,"num_object_copies":17650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5488,"num_write_kb":21952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5360},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8681","reported_seq":8923,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.148033+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:12.148033+0000","last_peered":"2024-09-16T04:52:12.148033+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:12.148033+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8681,"log_dups_size":0,"ondisk_log_size":8681,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35431634,"num_objects":8652,"num_object_clones":0,"num_object_copies":17304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8653,"num_write_kb":34603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5356}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8763","reported_seq":15619,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:12.134565+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:12.134565+0000","last_peered":"2024-09-16T04:52:12.134565+0000","last_clean":"2024-09-16T04:52:12.134565+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:12.134565+0000","last_undegraded":"2024-09-16T04:52:12.134565+0000","last_fullsized":"2024-09-16T04:52:12.134565+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8763,"log_dups_size":0,"ondisk_log_size":8763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35868672,"num_objects":8757,"num_object_clones":0,"num_object_copies":17514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5486,"num_write_kb":21944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8699","reported_seq":8797,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.125259+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:12.125259+0000","last_peered":"2024-09-16T04:52:12.125259+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:12.125259+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8699,"log_dups_size":0,"ondisk_log_size":8699,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35623529,"num_objects":8698,"num_object_clones":0,"num_object_copies":17396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8698,"num_write_kb":34789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'8970","reported_seq":9073,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.153041+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:12.153041+0000","last_peered":"2024-09-16T04:52:12.153041+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:12.153041+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":8970,"log_dups_size":0,"ondisk_log_size":8970,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36710203,"num_objects":8965,"num_object_clones":0,"num_object_copies":17930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8967,"num_write_kb":35852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":5451}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8830","reported_seq":10370,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.975538+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:12.975538+0000","last_peered":"2024-09-16T04:52:12.975538+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:12.975538+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8830,"log_dups_size":0,"ondisk_log_size":8830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2619,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5584,"num_write_kb":22336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":704,"num_bytes_recovered":2883584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6205},{"shards":"0,2","objects":2619}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'8945","reported_seq":9085,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:12.793921+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:12.793921+0000","last_peered":"2024-09-16T04:52:12.793921+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:12.793921+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":8945,"log_dups_size":0,"ondisk_log_size":8945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36590185,"num_objects":8934,"num_object_clones":0,"num_object_copies":17868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5714,"num_write_kb":22856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5410},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":575758559,"num_objects":140572,"num_object_clones":0,"num_object_copies":281144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2619,"num_objects_unfound":0,"num_objects_dirty":140572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105806,"num_write_kb":423196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4059,"num_bytes_recovered":16592896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1165762560,"data_stored":1165682543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140728,"ondisk_log_size":140728,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707460,"kb_used_data":432820,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93664380,"statfs":{"total":96636764160,"available":95912325120,"internally_reserved":0,"allocated":443207680,"data_stored":442938697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593760,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474428,"kb_used_data":271980,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93897412,"statfs":{"total":96636764160,"available":96150949888,"internally_reserved":0,"allocated":278507520,"data_stored":278168957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":207285934},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182451,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707756,"kb_used_data":433180,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93664084,"statfs":{"total":96636764160,"available":95912022016,"internally_reserved":0,"allocated":443576320,"data_stored":443307337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443457536,"data_stored":443454057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279207936,"data_stored":279134877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":443097088,"data_stored":443093609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:16.124 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 199 20 142224 142204 2.79028 1.88281 0.0129777 0.0279911 2024-09-16T04:52:17.124 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:2024-09-16T04:52:17.123126+0000 min lat: 0.00206892 max lat: 2.69414 avg lat: 0.0280382 2024-09-16T04:52:17.124 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T04:52:17.124 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 200 20 142692 142672 2.78547 1.82812 0.102072 0.0280382 2024-09-16T04:52:18.125 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 201 20 143128 143108 2.78009 1.70312 0.0332277 0.0280926 2024-09-16T04:52:18.441 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:19.124 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 202 20 143585 143565 2.77517 1.78516 0.0127498 0.0281424 2024-09-16T04:52:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:19.260 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:19.333 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-16T04:52:18.353330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":585347314,"num_objects":142915,"num_object_clones":0,"num_object_copies":285830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2539,"num_objects_unfound":0,"num_objects_dirty":142915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108149,"num_write_kb":432562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4140,"num_bytes_recovered":16920595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143072,"ondisk_log_size":143072,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1897516,"kb_used_data":1156924,"kb_used_omap":12,"kb_used_meta":740531,"kb_avail":281218004,"statfs":{"total":289910292480,"available":287967236096,"internally_reserved":0,"allocated":1184690176,"data_stored":1183813647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":758303745},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,4,6,1,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29171712,"num_objects":7122,"num_object_clones":0,"num_object_copies":14244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-268,"num_objects_unfound":0,"num_objects_dirty":7122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":28488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":1097728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006425"},"pg_stats":[{"pgid":"2.f","version":"47'8810","reported_seq":8889,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:18.026216+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:18.026216+0000","last_peered":"2024-09-16T04:52:18.026216+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:18.026216+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8810,"log_dups_size":0,"ondisk_log_size":8810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36085760,"num_objects":8810,"num_object_clones":0,"num_object_copies":17620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8810,"num_write_kb":35240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5469},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8903","reported_seq":9019,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:17.046840+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:17.046840+0000","last_peered":"2024-09-16T04:52:17.046840+0000","last_clean":"2024-09-16T04:52:17.046840+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:17.046840+0000","last_undegraded":"2024-09-16T04:52:17.046840+0000","last_fullsized":"2024-09-16T04:52:17.046840+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8903,"log_dups_size":0,"ondisk_log_size":8903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36442112,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5872,"num_write_kb":23488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8837","reported_seq":8975,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.154775+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:17.154775+0000","last_peered":"2024-09-16T04:52:17.154775+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:17.154775+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8837,"log_dups_size":0,"ondisk_log_size":8837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36179968,"num_objects":8833,"num_object_clones":0,"num_object_copies":17666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5576,"num_write_kb":22304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9013","reported_seq":9083,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:17.127130+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:17.127130+0000","last_peered":"2024-09-16T04:52:17.127130+0000","last_clean":"2024-09-16T04:52:17.127130+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:17.127130+0000","last_undegraded":"2024-09-16T04:52:17.127130+0000","last_fullsized":"2024-09-16T04:52:17.127130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9013,"log_dups_size":0,"ondisk_log_size":9013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36917248,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9013,"num_write_kb":36052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8977","reported_seq":9196,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:18.024631+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:18.024631+0000","last_peered":"2024-09-16T04:52:18.024631+0000","last_clean":"2024-09-16T04:52:18.024631+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:18.024631+0000","last_undegraded":"2024-09-16T04:52:18.024631+0000","last_fullsized":"2024-09-16T04:52:18.024631+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8977,"log_dups_size":0,"ondisk_log_size":8977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36646912,"num_objects":8947,"num_object_clones":0,"num_object_copies":17894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6009,"num_write_kb":24036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9018","reported_seq":9150,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.916330+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:17.916330+0000","last_peered":"2024-09-16T04:52:17.916330+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:17.916330+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9018,"log_dups_size":0,"ondisk_log_size":9018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36917248,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6011,"num_write_kb":24044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5557},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8869","reported_seq":9017,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:18.019127+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:18.019127+0000","last_peered":"2024-09-16T04:52:18.019127+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:18.019127+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8869,"log_dups_size":0,"ondisk_log_size":8869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36302848,"num_objects":8863,"num_object_clones":0,"num_object_copies":17726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5524,"num_write_kb":22096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5464},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8922","reported_seq":9138,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:17.840859+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:17.840859+0000","last_peered":"2024-09-16T04:52:17.840859+0000","last_clean":"2024-09-16T04:52:17.840859+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:17.840859+0000","last_undegraded":"2024-09-16T04:52:17.840859+0000","last_fullsized":"2024-09-16T04:52:17.840859+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8922,"log_dups_size":0,"ondisk_log_size":8922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36511744,"num_objects":8914,"num_object_clones":0,"num_object_copies":17828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5992,"num_write_kb":23968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8982","reported_seq":9210,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.093243+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:17.093243+0000","last_peered":"2024-09-16T04:52:17.093243+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:17.093243+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8982,"log_dups_size":0,"ondisk_log_size":8982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36655104,"num_objects":8949,"num_object_clones":0,"num_object_copies":17898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5733,"num_write_kb":22932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5628}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8975","reported_seq":9105,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.952191+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:17.952191+0000","last_peered":"2024-09-16T04:52:17.952191+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:17.952191+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8975,"log_dups_size":0,"ondisk_log_size":8975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36728832,"num_objects":8967,"num_object_clones":0,"num_object_copies":17934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5630,"num_write_kb":22520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5502},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8812","reported_seq":9054,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.177333+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:17.177333+0000","last_peered":"2024-09-16T04:52:17.177333+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:17.177333+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8812,"log_dups_size":0,"ondisk_log_size":8812,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35968210,"num_objects":8783,"num_object_clones":0,"num_object_copies":17566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8784,"num_write_kb":35127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5487}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8903","reported_seq":15758,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:17.210045+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:17.210045+0000","last_peered":"2024-09-16T04:52:17.210045+0000","last_clean":"2024-09-16T04:52:17.210045+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:17.210045+0000","last_undegraded":"2024-09-16T04:52:17.210045+0000","last_fullsized":"2024-09-16T04:52:17.210045+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8903,"log_dups_size":0,"ondisk_log_size":8903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36442112,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5626,"num_write_kb":22504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8832","reported_seq":8930,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:16.971953+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:16.971953+0000","last_peered":"2024-09-16T04:52:16.971953+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:16.971953+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8832,"log_dups_size":0,"ondisk_log_size":8832,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36168297,"num_objects":8831,"num_object_clones":0,"num_object_copies":17662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8831,"num_write_kb":35321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5424}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9132","reported_seq":9235,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.033459+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:17.033459+0000","last_peered":"2024-09-16T04:52:17.033459+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:17.033459+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9132,"log_dups_size":0,"ondisk_log_size":9132,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37373755,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9129,"num_write_kb":36500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":5613}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8976","reported_seq":10676,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:18.063378+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:18.063378+0000","last_peered":"2024-09-16T04:52:18.063378+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:18.063378+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8976,"log_dups_size":0,"ondisk_log_size":8976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36749312,"num_objects":8972,"num_object_clones":0,"num_object_copies":17944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2539,"num_objects_unfound":0,"num_objects_dirty":8972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5730,"num_write_kb":22920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":3211264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6431},{"shards":"0,2","objects":2539}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9108","reported_seq":9248,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:18.010593+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:18.010593+0000","last_peered":"2024-09-16T04:52:18.010593+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:18.010593+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9108,"log_dups_size":0,"ondisk_log_size":9108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257833,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5877,"num_write_kb":23508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5573},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":585347295,"num_objects":142913,"num_object_clones":0,"num_object_copies":285826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2539,"num_objects_unfound":0,"num_objects_dirty":142913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108147,"num_write_kb":432560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":16920576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1185484800,"data_stored":1185404783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143069,"ondisk_log_size":143069,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705076,"kb_used_data":438756,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93666764,"statfs":{"total":96636764160,"available":95914766336,"internally_reserved":0,"allocated":449286144,"data_stored":449017161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":272690857},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,6,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593761,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491320,"kb_used_data":279144,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93880520,"statfs":{"total":96636764160,"available":96133652480,"internally_reserved":0,"allocated":285843456,"data_stored":285504893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701120,"kb_used_data":439024,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93670720,"statfs":{"total":96636764160,"available":95918817280,"internally_reserved":0,"allocated":449560576,"data_stored":449291593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,0,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":449568768,"data_stored":449565289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":286756864,"data_stored":286683805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":449159168,"data_stored":449155689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:19.335 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:20.082 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:20.083 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:20.124 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 203 20 144043 144023 2.77031 1.78906 0.0228024 0.0281925 2024-09-16T04:52:20.149 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-16T04:52:18.353330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":585347314,"num_objects":142915,"num_object_clones":0,"num_object_copies":285830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2539,"num_objects_unfound":0,"num_objects_dirty":142915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108149,"num_write_kb":432562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4140,"num_bytes_recovered":16920595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143072,"ondisk_log_size":143072,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1897516,"kb_used_data":1156924,"kb_used_omap":12,"kb_used_meta":740531,"kb_avail":281218004,"statfs":{"total":289910292480,"available":287967236096,"internally_reserved":0,"allocated":1184690176,"data_stored":1183813647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":758303745},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,4,6,1,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29171712,"num_objects":7122,"num_object_clones":0,"num_object_copies":14244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-268,"num_objects_unfound":0,"num_objects_dirty":7122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":28488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":1097728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006425"},"pg_stats":[{"pgid":"2.f","version":"47'8810","reported_seq":8889,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:18.026216+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:18.026216+0000","last_peered":"2024-09-16T04:52:18.026216+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:18.026216+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8810,"log_dups_size":0,"ondisk_log_size":8810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36085760,"num_objects":8810,"num_object_clones":0,"num_object_copies":17620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8810,"num_write_kb":35240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5469},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8903","reported_seq":9019,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:17.046840+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:17.046840+0000","last_peered":"2024-09-16T04:52:17.046840+0000","last_clean":"2024-09-16T04:52:17.046840+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:17.046840+0000","last_undegraded":"2024-09-16T04:52:17.046840+0000","last_fullsized":"2024-09-16T04:52:17.046840+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8903,"log_dups_size":0,"ondisk_log_size":8903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36442112,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5872,"num_write_kb":23488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8837","reported_seq":8975,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.154775+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:17.154775+0000","last_peered":"2024-09-16T04:52:17.154775+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:17.154775+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8837,"log_dups_size":0,"ondisk_log_size":8837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36179968,"num_objects":8833,"num_object_clones":0,"num_object_copies":17666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5576,"num_write_kb":22304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9013","reported_seq":9083,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:17.127130+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:17.127130+0000","last_peered":"2024-09-16T04:52:17.127130+0000","last_clean":"2024-09-16T04:52:17.127130+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:17.127130+0000","last_undegraded":"2024-09-16T04:52:17.127130+0000","last_fullsized":"2024-09-16T04:52:17.127130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9013,"log_dups_size":0,"ondisk_log_size":9013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36917248,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9013,"num_write_kb":36052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8977","reported_seq":9196,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:18.024631+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:18.024631+0000","last_peered":"2024-09-16T04:52:18.024631+0000","last_clean":"2024-09-16T04:52:18.024631+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:18.024631+0000","last_undegraded":"2024-09-16T04:52:18.024631+0000","last_fullsized":"2024-09-16T04:52:18.024631+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8977,"log_dups_size":0,"ondisk_log_size":8977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36646912,"num_objects":8947,"num_object_clones":0,"num_object_copies":17894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6009,"num_write_kb":24036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9018","reported_seq":9150,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.916330+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:17.916330+0000","last_peered":"2024-09-16T04:52:17.916330+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:17.916330+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9018,"log_dups_size":0,"ondisk_log_size":9018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36917248,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6011,"num_write_kb":24044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5557},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8869","reported_seq":9017,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:18.019127+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:18.019127+0000","last_peered":"2024-09-16T04:52:18.019127+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:18.019127+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8869,"log_dups_size":0,"ondisk_log_size":8869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36302848,"num_objects":8863,"num_object_clones":0,"num_object_copies":17726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5524,"num_write_kb":22096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5464},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8922","reported_seq":9138,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:17.840859+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:17.840859+0000","last_peered":"2024-09-16T04:52:17.840859+0000","last_clean":"2024-09-16T04:52:17.840859+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:17.840859+0000","last_undegraded":"2024-09-16T04:52:17.840859+0000","last_fullsized":"2024-09-16T04:52:17.840859+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8922,"log_dups_size":0,"ondisk_log_size":8922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36511744,"num_objects":8914,"num_object_clones":0,"num_object_copies":17828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5992,"num_write_kb":23968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8982","reported_seq":9210,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.093243+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:17.093243+0000","last_peered":"2024-09-16T04:52:17.093243+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:17.093243+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8982,"log_dups_size":0,"ondisk_log_size":8982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36655104,"num_objects":8949,"num_object_clones":0,"num_object_copies":17898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5733,"num_write_kb":22932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5628}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8975","reported_seq":9105,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.952191+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:17.952191+0000","last_peered":"2024-09-16T04:52:17.952191+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:17.952191+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8975,"log_dups_size":0,"ondisk_log_size":8975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36728832,"num_objects":8967,"num_object_clones":0,"num_object_copies":17934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5630,"num_write_kb":22520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5502},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8812","reported_seq":9054,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.177333+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:17.177333+0000","last_peered":"2024-09-16T04:52:17.177333+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:17.177333+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8812,"log_dups_size":0,"ondisk_log_size":8812,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35968210,"num_objects":8783,"num_object_clones":0,"num_object_copies":17566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8784,"num_write_kb":35127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5487}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8903","reported_seq":15758,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:17.210045+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:17.210045+0000","last_peered":"2024-09-16T04:52:17.210045+0000","last_clean":"2024-09-16T04:52:17.210045+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:17.210045+0000","last_undegraded":"2024-09-16T04:52:17.210045+0000","last_fullsized":"2024-09-16T04:52:17.210045+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8903,"log_dups_size":0,"ondisk_log_size":8903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36442112,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5626,"num_write_kb":22504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8832","reported_seq":8930,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:16.971953+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:16.971953+0000","last_peered":"2024-09-16T04:52:16.971953+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:16.971953+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8832,"log_dups_size":0,"ondisk_log_size":8832,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36168297,"num_objects":8831,"num_object_clones":0,"num_object_copies":17662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8831,"num_write_kb":35321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5424}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9132","reported_seq":9235,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.033459+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:17.033459+0000","last_peered":"2024-09-16T04:52:17.033459+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:17.033459+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9132,"log_dups_size":0,"ondisk_log_size":9132,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37373755,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9129,"num_write_kb":36500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":5613}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8976","reported_seq":10676,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:18.063378+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:18.063378+0000","last_peered":"2024-09-16T04:52:18.063378+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:18.063378+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8976,"log_dups_size":0,"ondisk_log_size":8976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36749312,"num_objects":8972,"num_object_clones":0,"num_object_copies":17944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2539,"num_objects_unfound":0,"num_objects_dirty":8972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5730,"num_write_kb":22920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":3211264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6431},{"shards":"0,2","objects":2539}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9108","reported_seq":9248,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:18.010593+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:18.010593+0000","last_peered":"2024-09-16T04:52:18.010593+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:18.010593+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9108,"log_dups_size":0,"ondisk_log_size":9108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257833,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5877,"num_write_kb":23508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5573},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":585347295,"num_objects":142913,"num_object_clones":0,"num_object_copies":285826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2539,"num_objects_unfound":0,"num_objects_dirty":142913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108147,"num_write_kb":432560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":16920576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1185484800,"data_stored":1185404783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143069,"ondisk_log_size":143069,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705076,"kb_used_data":438756,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93666764,"statfs":{"total":96636764160,"available":95914766336,"internally_reserved":0,"allocated":449286144,"data_stored":449017161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":272690857},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,6,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593761,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491320,"kb_used_data":279144,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93880520,"statfs":{"total":96636764160,"available":96133652480,"internally_reserved":0,"allocated":285843456,"data_stored":285504893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701120,"kb_used_data":439024,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93670720,"statfs":{"total":96636764160,"available":95918817280,"internally_reserved":0,"allocated":449560576,"data_stored":449291593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,0,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":449568768,"data_stored":449565289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":286756864,"data_stored":286683805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":449159168,"data_stored":449155689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:20.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:52:20.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'8810", 'reported_seq': 8889, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:18.026216+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:52:18.026216+0000', 'last_peered': '2024-09-16T04:52:18.026216+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:52:18.026216+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 8810, 'log_dups_size': 0, 'ondisk_log_size': 8810, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36085760, 'num_objects': 8810, 'num_object_clones': 0, 'num_object_copies': 17620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8810, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8810, 'num_write_kb': 35240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5469}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:20.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:52:20.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'8837", 'reported_seq': 8975, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:17.154775+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:52:17.154775+0000', 'last_peered': '2024-09-16T04:52:17.154775+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:52:17.154775+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 8837, 'log_dups_size': 0, 'ondisk_log_size': 8837, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36179968, 'num_objects': 8833, 'num_object_clones': 0, 'num_object_copies': 17666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8833, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5576, 'num_write_kb': 22304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 5514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:20.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:52:20.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'9018", 'reported_seq': 9150, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:17.916330+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:52:17.916330+0000', 'last_peered': '2024-09-16T04:52:17.916330+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:52:17.916330+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 9018, 'log_dups_size': 0, 'ondisk_log_size': 9018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36917248, 'num_objects': 9013, 'num_object_clones': 0, 'num_object_copies': 18026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9013, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6011, 'num_write_kb': 24044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5557}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:20.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:52:20.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'8869", 'reported_seq': 9017, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:18.019127+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:52:18.019127+0000', 'last_peered': '2024-09-16T04:52:18.019127+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:52:18.019127+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 8869, 'log_dups_size': 0, 'ondisk_log_size': 8869, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36302848, 'num_objects': 8863, 'num_object_clones': 0, 'num_object_copies': 17726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8863, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5524, 'num_write_kb': 22096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5464}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:20.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:52:20.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'8982", 'reported_seq': 9210, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:17.093243+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:52:17.093243+0000', 'last_peered': '2024-09-16T04:52:17.093243+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:52:17.093243+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 8982, 'log_dups_size': 0, 'ondisk_log_size': 8982, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36655104, 'num_objects': 8949, 'num_object_clones': 0, 'num_object_copies': 17898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8949, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5733, 'num_write_kb': 22932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 5628}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:20.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:52:20.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'8975", 'reported_seq': 9105, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:17.952191+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:52:17.952191+0000', 'last_peered': '2024-09-16T04:52:17.952191+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:52:17.952191+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 8975, 'log_dups_size': 0, 'ondisk_log_size': 8975, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36728832, 'num_objects': 8967, 'num_object_clones': 0, 'num_object_copies': 17934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8967, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5630, 'num_write_kb': 22520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5502}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:20.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:52:20.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'8812", 'reported_seq': 9054, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:17.177333+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:52:17.177333+0000', 'last_peered': '2024-09-16T04:52:17.177333+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:52:17.177333+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 8812, 'log_dups_size': 0, 'ondisk_log_size': 8812, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35968210, 'num_objects': 8783, 'num_object_clones': 0, 'num_object_copies': 17566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8783, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8784, 'num_write_kb': 35127, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 5487}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:20.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:52:20.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'8832", 'reported_seq': 8930, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:16.971953+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:52:16.971953+0000', 'last_peered': '2024-09-16T04:52:16.971953+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:52:16.971953+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 8832, 'log_dups_size': 0, 'ondisk_log_size': 8832, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36168297, 'num_objects': 8831, 'num_object_clones': 0, 'num_object_copies': 17662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8831, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8831, 'num_write_kb': 35321, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 5424}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:20.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:52:20.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'9132", 'reported_seq': 9235, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:17.033459+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:52:17.033459+0000', 'last_peered': '2024-09-16T04:52:17.033459+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:52:17.033459+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 9132, 'log_dups_size': 0, 'ondisk_log_size': 9132, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37373755, 'num_objects': 9127, 'num_object_clones': 0, 'num_object_copies': 18254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9129, 'num_write_kb': 36500, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 5613}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:20.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:52:20.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'8976", 'reported_seq': 10676, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:18.063378+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:52:18.063378+0000', 'last_peered': '2024-09-16T04:52:18.063378+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:52:18.063378+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 8976, 'log_dups_size': 0, 'ondisk_log_size': 8976, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36749312, 'num_objects': 8972, 'num_object_clones': 0, 'num_object_copies': 17944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2539, 'num_objects_unfound': 0, 'num_objects_dirty': 8972, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5730, 'num_write_kb': 22920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 784, 'num_bytes_recovered': 3211264, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6431}, {'shards': '0,2', 'objects': 2539}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:20.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:52:20.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'9108", 'reported_seq': 9248, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:18.010593+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:52:18.010593+0000', 'last_peered': '2024-09-16T04:52:18.010593+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:52:18.010593+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 9108, 'log_dups_size': 0, 'ondisk_log_size': 9108, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37257833, 'num_objects': 9097, 'num_object_clones': 0, 'num_object_copies': 18194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9097, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5877, 'num_write_kb': 23508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5573}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:20.152 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:52:21.124 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 204 20 144504 144484 2.76556 1.80078 0.0659024 0.0282415 2024-09-16T04:52:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/285830 objects degraded (11.917%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":289,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":142915,"data_bytes":585347314,"bytes_used":1943056384,"bytes_avail":287967236096,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":285830,"degraded_ratio":0.11916873666165204,"misplaced_objects":2539,"misplaced_total":285830,"misplaced_ratio":0.0088829024245180698,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":59011,"recovering_keys_per_sec":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"write_bytes_sec":1619907,"read_op_per_sec":0,"write_op_per_sec":395},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:52:21.366 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:52:21.367 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:22.129 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 205 20 144934 144914 2.76019 1.67969 0.0844065 0.0282946 2024-09-16T04:52:22.170 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:22.171 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:22.226 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-16T04:52:20.355408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":585347314,"num_objects":142915,"num_object_clones":0,"num_object_copies":285830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2539,"num_objects_unfound":0,"num_objects_dirty":142915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108149,"num_write_kb":432562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4140,"num_bytes_recovered":16920595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143072,"ondisk_log_size":143072,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1897516,"kb_used_data":1156924,"kb_used_omap":12,"kb_used_meta":740531,"kb_avail":281218004,"statfs":{"total":289910292480,"available":287967236096,"internally_reserved":0,"allocated":1184690176,"data_stored":1183813647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":758303745},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,4,6,1,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19451904,"num_objects":4749,"num_object_clones":0,"num_object_copies":9498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":4749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4749,"num_write_kb":18996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008037"},"pg_stats":[{"pgid":"2.f","version":"47'8810","reported_seq":8889,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:18.026216+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:18.026216+0000","last_peered":"2024-09-16T04:52:18.026216+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:18.026216+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8810,"log_dups_size":0,"ondisk_log_size":8810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36085760,"num_objects":8810,"num_object_clones":0,"num_object_copies":17620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8810,"num_write_kb":35240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5469},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'8903","reported_seq":9019,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:17.046840+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:17.046840+0000","last_peered":"2024-09-16T04:52:17.046840+0000","last_clean":"2024-09-16T04:52:17.046840+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:17.046840+0000","last_undegraded":"2024-09-16T04:52:17.046840+0000","last_fullsized":"2024-09-16T04:52:17.046840+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":8903,"log_dups_size":0,"ondisk_log_size":8903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36442112,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5872,"num_write_kb":23488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8837","reported_seq":8975,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.154775+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:17.154775+0000","last_peered":"2024-09-16T04:52:17.154775+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:17.154775+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8837,"log_dups_size":0,"ondisk_log_size":8837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36179968,"num_objects":8833,"num_object_clones":0,"num_object_copies":17666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5576,"num_write_kb":22304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9013","reported_seq":9083,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:17.127130+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:17.127130+0000","last_peered":"2024-09-16T04:52:17.127130+0000","last_clean":"2024-09-16T04:52:17.127130+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:17.127130+0000","last_undegraded":"2024-09-16T04:52:17.127130+0000","last_fullsized":"2024-09-16T04:52:17.127130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9013,"log_dups_size":0,"ondisk_log_size":9013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36917248,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9013,"num_write_kb":36052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'8977","reported_seq":9196,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:18.024631+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:18.024631+0000","last_peered":"2024-09-16T04:52:18.024631+0000","last_clean":"2024-09-16T04:52:18.024631+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:18.024631+0000","last_undegraded":"2024-09-16T04:52:18.024631+0000","last_fullsized":"2024-09-16T04:52:18.024631+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":8977,"log_dups_size":0,"ondisk_log_size":8977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36646912,"num_objects":8947,"num_object_clones":0,"num_object_copies":17894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6009,"num_write_kb":24036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9018","reported_seq":9150,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.916330+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:17.916330+0000","last_peered":"2024-09-16T04:52:17.916330+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:17.916330+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9018,"log_dups_size":0,"ondisk_log_size":9018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36917248,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6011,"num_write_kb":24044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5557},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'8869","reported_seq":9017,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:18.019127+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:18.019127+0000","last_peered":"2024-09-16T04:52:18.019127+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:18.019127+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":8869,"log_dups_size":0,"ondisk_log_size":8869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36302848,"num_objects":8863,"num_object_clones":0,"num_object_copies":17726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5524,"num_write_kb":22096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5464},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'8922","reported_seq":9138,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:17.840859+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:17.840859+0000","last_peered":"2024-09-16T04:52:17.840859+0000","last_clean":"2024-09-16T04:52:17.840859+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:17.840859+0000","last_undegraded":"2024-09-16T04:52:17.840859+0000","last_fullsized":"2024-09-16T04:52:17.840859+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":8922,"log_dups_size":0,"ondisk_log_size":8922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36511744,"num_objects":8914,"num_object_clones":0,"num_object_copies":17828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5992,"num_write_kb":23968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'8982","reported_seq":9210,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.093243+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:17.093243+0000","last_peered":"2024-09-16T04:52:17.093243+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:17.093243+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":8982,"log_dups_size":0,"ondisk_log_size":8982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36655104,"num_objects":8949,"num_object_clones":0,"num_object_copies":17898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5733,"num_write_kb":22932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5628}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'8975","reported_seq":9105,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.952191+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:17.952191+0000","last_peered":"2024-09-16T04:52:17.952191+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:17.952191+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":8975,"log_dups_size":0,"ondisk_log_size":8975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36728832,"num_objects":8967,"num_object_clones":0,"num_object_copies":17934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5630,"num_write_kb":22520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5502},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8812","reported_seq":9054,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.177333+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:17.177333+0000","last_peered":"2024-09-16T04:52:17.177333+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:17.177333+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8812,"log_dups_size":0,"ondisk_log_size":8812,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35968210,"num_objects":8783,"num_object_clones":0,"num_object_copies":17566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8784,"num_write_kb":35127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5487}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'8903","reported_seq":15758,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:17.210045+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:17.210045+0000","last_peered":"2024-09-16T04:52:17.210045+0000","last_clean":"2024-09-16T04:52:17.210045+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:17.210045+0000","last_undegraded":"2024-09-16T04:52:17.210045+0000","last_fullsized":"2024-09-16T04:52:17.210045+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":8903,"log_dups_size":0,"ondisk_log_size":8903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36442112,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5626,"num_write_kb":22504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8832","reported_seq":8930,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:16.971953+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:16.971953+0000","last_peered":"2024-09-16T04:52:16.971953+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:16.971953+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8832,"log_dups_size":0,"ondisk_log_size":8832,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36168297,"num_objects":8831,"num_object_clones":0,"num_object_copies":17662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8831,"num_write_kb":35321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5424}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9132","reported_seq":9235,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:17.033459+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:17.033459+0000","last_peered":"2024-09-16T04:52:17.033459+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:17.033459+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9132,"log_dups_size":0,"ondisk_log_size":9132,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37373755,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9129,"num_write_kb":36500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":5613}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'8976","reported_seq":10676,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:18.063378+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:18.063378+0000","last_peered":"2024-09-16T04:52:18.063378+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:18.063378+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":8976,"log_dups_size":0,"ondisk_log_size":8976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36749312,"num_objects":8972,"num_object_clones":0,"num_object_copies":17944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2539,"num_objects_unfound":0,"num_objects_dirty":8972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5730,"num_write_kb":22920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":784,"num_bytes_recovered":3211264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6431},{"shards":"0,2","objects":2539}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9108","reported_seq":9248,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:18.010593+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:18.010593+0000","last_peered":"2024-09-16T04:52:18.010593+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:18.010593+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9108,"log_dups_size":0,"ondisk_log_size":9108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257833,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5877,"num_write_kb":23508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5573},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":585347295,"num_objects":142913,"num_object_clones":0,"num_object_copies":285826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2539,"num_objects_unfound":0,"num_objects_dirty":142913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108147,"num_write_kb":432560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4139,"num_bytes_recovered":16920576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1185484800,"data_stored":1185404783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143069,"ondisk_log_size":143069,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":705076,"kb_used_data":438756,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93666764,"statfs":{"total":96636764160,"available":95914766336,"internally_reserved":0,"allocated":449286144,"data_stored":449017161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":272690857},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,6,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593761,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491320,"kb_used_data":279144,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93880520,"statfs":{"total":96636764160,"available":96133652480,"internally_reserved":0,"allocated":285843456,"data_stored":285504893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":217247406},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701120,"kb_used_data":439024,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93670720,"statfs":{"total":96636764160,"available":95918817280,"internally_reserved":0,"allocated":449560576,"data_stored":449291593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,0,0,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":449568768,"data_stored":449565289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":286756864,"data_stored":286683805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":449159168,"data_stored":449155689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:23.129 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 206 20 145418 145398 2.75597 1.89062 0.0111508 0.0283366 2024-09-16T04:52:24.129 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 207 20 145899 145879 2.75174 1.87891 0.014577 0.0283831 2024-09-16T04:52:25.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 208 20 146393 146373 2.74779 1.92969 0.0174073 0.0284243 2024-09-16T04:52:25.229 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:26.112 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:26.113 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:26.129 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 209 20 146847 146827 2.74312 1.77344 0.0153988 0.0284712 2024-09-16T04:52:26.173 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-16T04:52:24.357278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594649330,"num_objects":145186,"num_object_clones":0,"num_object_copies":290372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2446,"num_objects_unfound":0,"num_objects_dirty":145186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110420,"num_write_kb":441646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4233,"num_bytes_recovered":17301523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145343,"ondisk_log_size":145343,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1946472,"kb_used_data":1175352,"kb_used_omap":12,"kb_used_meta":771059,"kb_avail":281169048,"statfs":{"total":289910292480,"available":287917105152,"internally_reserved":0,"allocated":1203560448,"data_stored":1202683919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":789564417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,4,5,8,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5811,"num_write_kb":23244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008943"},"pg_stats":[{"pgid":"2.f","version":"47'8948","reported_seq":9027,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:22.968146+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:22.968146+0000","last_peered":"2024-09-16T04:52:22.968146+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:22.968146+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8948,"log_dups_size":0,"ondisk_log_size":8948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36651008,"num_objects":8948,"num_object_clones":0,"num_object_copies":17896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":35792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5607},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9049","reported_seq":9165,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:22.224576+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:22.224576+0000","last_peered":"2024-09-16T04:52:22.224576+0000","last_clean":"2024-09-16T04:52:22.224576+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:22.224576+0000","last_undegraded":"2024-09-16T04:52:22.224576+0000","last_fullsized":"2024-09-16T04:52:22.224576+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9049,"log_dups_size":0,"ondisk_log_size":9049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37040128,"num_objects":9043,"num_object_clones":0,"num_object_copies":18086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6018,"num_write_kb":24072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8963","reported_seq":9101,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:22.137071+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:22.137071+0000","last_peered":"2024-09-16T04:52:22.137071+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:22.137071+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8963,"log_dups_size":0,"ondisk_log_size":8963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36696064,"num_objects":8959,"num_object_clones":0,"num_object_copies":17918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5702,"num_write_kb":22808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5640}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9155","reported_seq":9225,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:22.157917+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:22.157917+0000","last_peered":"2024-09-16T04:52:22.157917+0000","last_clean":"2024-09-16T04:52:22.157917+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:22.157917+0000","last_undegraded":"2024-09-16T04:52:22.157917+0000","last_fullsized":"2024-09-16T04:52:22.157917+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9155,"log_dups_size":0,"ondisk_log_size":9155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37498880,"num_objects":9155,"num_object_clones":0,"num_object_copies":18310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9155,"num_write_kb":36620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9125","reported_seq":9344,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:22.956200+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:22.956200+0000","last_peered":"2024-09-16T04:52:22.956200+0000","last_clean":"2024-09-16T04:52:22.956200+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:22.956200+0000","last_undegraded":"2024-09-16T04:52:22.956200+0000","last_fullsized":"2024-09-16T04:52:22.956200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9125,"log_dups_size":0,"ondisk_log_size":9125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37253120,"num_objects":9095,"num_object_clones":0,"num_object_copies":18190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6157,"num_write_kb":24628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9163","reported_seq":9295,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:23.032560+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:23.032560+0000","last_peered":"2024-09-16T04:52:23.032560+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:23.032560+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9163,"log_dups_size":0,"ondisk_log_size":9163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37511168,"num_objects":9158,"num_object_clones":0,"num_object_copies":18316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6156,"num_write_kb":24624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5702},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9009","reported_seq":9157,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:23.007868+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:23.007868+0000","last_peered":"2024-09-16T04:52:23.007868+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:23.007868+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9009,"log_dups_size":0,"ondisk_log_size":9009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36876288,"num_objects":9003,"num_object_clones":0,"num_object_copies":18006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5664,"num_write_kb":22656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5604},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9052","reported_seq":9267,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:23.064981+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:23.064981+0000","last_peered":"2024-09-16T04:52:23.064981+0000","last_clean":"2024-09-16T04:52:23.064981+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:23.064981+0000","last_undegraded":"2024-09-16T04:52:23.064981+0000","last_fullsized":"2024-09-16T04:52:23.064981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9052,"log_dups_size":0,"ondisk_log_size":9052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37044224,"num_objects":9044,"num_object_clones":0,"num_object_copies":18088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6122,"num_write_kb":24488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9108","reported_seq":9336,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:22.180475+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:22.180475+0000","last_peered":"2024-09-16T04:52:22.180475+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:22.180475+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9108,"log_dups_size":0,"ondisk_log_size":9108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37171200,"num_objects":9075,"num_object_clones":0,"num_object_copies":18150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5859,"num_write_kb":23436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5754}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'9138","reported_seq":9268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:23.011108+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:23.011108+0000","last_peered":"2024-09-16T04:52:23.011108+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:23.011108+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":9138,"log_dups_size":0,"ondisk_log_size":9138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37396480,"num_objects":9130,"num_object_clones":0,"num_object_copies":18260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5793,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5665},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8973","reported_seq":9215,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:22.204848+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:22.204848+0000","last_peered":"2024-09-16T04:52:22.204848+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:22.204848+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8973,"log_dups_size":0,"ondisk_log_size":8973,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36627666,"num_objects":8944,"num_object_clones":0,"num_object_copies":17888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8945,"num_write_kb":35771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5648}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9014","reported_seq":15870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:22.004210+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:22.004210+0000","last_peered":"2024-09-16T04:52:22.004210+0000","last_clean":"2024-09-16T04:52:22.004210+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:22.004210+0000","last_undegraded":"2024-09-16T04:52:22.004210+0000","last_fullsized":"2024-09-16T04:52:22.004210+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9014,"log_dups_size":0,"ondisk_log_size":9014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36896768,"num_objects":9008,"num_object_clones":0,"num_object_copies":18016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5737,"num_write_kb":22948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8983","reported_seq":9079,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:22.224355+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:22.224355+0000","last_peered":"2024-09-16T04:52:22.224355+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:22.224355+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8983,"log_dups_size":0,"ondisk_log_size":8983,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36786793,"num_objects":8982,"num_object_clones":0,"num_object_copies":17964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8982,"num_write_kb":35925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5575}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9280","reported_seq":9383,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:22.170871+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:22.170871+0000","last_peered":"2024-09-16T04:52:22.170871+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:22.170871+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9280,"log_dups_size":0,"ondisk_log_size":9280,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37979963,"num_objects":9275,"num_object_clones":0,"num_object_copies":18550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9277,"num_write_kb":37092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":5761}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9145","reported_seq":11031,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:23.093932+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:23.093932+0000","last_peered":"2024-09-16T04:52:23.093932+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:23.093932+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9145,"log_dups_size":0,"ondisk_log_size":9145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37441536,"num_objects":9141,"num_object_clones":0,"num_object_copies":18282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2446,"num_objects_unfound":0,"num_objects_dirty":9141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5899,"num_write_kb":23596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":877,"num_bytes_recovered":3592192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6693},{"shards":"0,2","objects":2446}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9235","reported_seq":9375,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:23.077758+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:23.077758+0000","last_peered":"2024-09-16T04:52:23.077758+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:23.077758+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9235,"log_dups_size":0,"ondisk_log_size":9235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37778025,"num_objects":9224,"num_object_clones":0,"num_object_copies":18448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6004,"num_write_kb":24016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5700},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":594649311,"num_objects":145184,"num_object_clones":0,"num_object_copies":290368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2446,"num_objects_unfound":0,"num_objects_dirty":145184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110418,"num_write_kb":441644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4232,"num_bytes_recovered":17301504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204518912,"data_stored":1204438895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145340,"ondisk_log_size":145340,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":714532,"kb_used_data":444052,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93657308,"statfs":{"total":96636764160,"available":95905083392,"internally_reserved":0,"allocated":454709248,"data_stored":454440265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":276950697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,1,3,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593762,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502832,"kb_used_data":286496,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93869008,"statfs":{"total":96636764160,"available":96121864192,"internally_reserved":0,"allocated":293371904,"data_stored":293033341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":221507246},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729108,"kb_used_data":444804,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93642732,"statfs":{"total":96636764160,"available":95890157568,"internally_reserved":0,"allocated":455479296,"data_stored":455210313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":455503872,"data_stored":455500393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294219776,"data_stored":294146717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454795264,"data_stored":454791785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:26.175 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:27.123 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:27.123 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:27.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 210 20 147295 147275 2.73837 1.75 0.0426063 0.0285221 2024-09-16T04:52:27.185 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-16T04:52:26.357764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594649330,"num_objects":145186,"num_object_clones":0,"num_object_copies":290372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2446,"num_objects_unfound":0,"num_objects_dirty":145186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110420,"num_write_kb":441646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4233,"num_bytes_recovered":17301523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145343,"ondisk_log_size":145343,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1976052,"kb_used_data":1182724,"kb_used_omap":12,"kb_used_meta":793267,"kb_avail":281139468,"statfs":{"total":289910292480,"available":287886815232,"internally_reserved":0,"allocated":1211109376,"data_stored":1210232847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":812305409},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,4,4,8,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18890752,"num_objects":4612,"num_object_clones":0,"num_object_copies":9224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-173,"num_objects_unfound":0,"num_objects_dirty":4612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4612,"num_write_kb":18448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008974"},"pg_stats":[{"pgid":"2.f","version":"47'8948","reported_seq":9027,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:22.968146+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:22.968146+0000","last_peered":"2024-09-16T04:52:22.968146+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:22.968146+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8948,"log_dups_size":0,"ondisk_log_size":8948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36651008,"num_objects":8948,"num_object_clones":0,"num_object_copies":17896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":35792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5607},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9049","reported_seq":9165,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:22.224576+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:22.224576+0000","last_peered":"2024-09-16T04:52:22.224576+0000","last_clean":"2024-09-16T04:52:22.224576+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:22.224576+0000","last_undegraded":"2024-09-16T04:52:22.224576+0000","last_fullsized":"2024-09-16T04:52:22.224576+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9049,"log_dups_size":0,"ondisk_log_size":9049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37040128,"num_objects":9043,"num_object_clones":0,"num_object_copies":18086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6018,"num_write_kb":24072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'8963","reported_seq":9101,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:22.137071+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:22.137071+0000","last_peered":"2024-09-16T04:52:22.137071+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:22.137071+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":8963,"log_dups_size":0,"ondisk_log_size":8963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36696064,"num_objects":8959,"num_object_clones":0,"num_object_copies":17918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5702,"num_write_kb":22808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5640}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9155","reported_seq":9225,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:22.157917+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:22.157917+0000","last_peered":"2024-09-16T04:52:22.157917+0000","last_clean":"2024-09-16T04:52:22.157917+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:22.157917+0000","last_undegraded":"2024-09-16T04:52:22.157917+0000","last_fullsized":"2024-09-16T04:52:22.157917+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9155,"log_dups_size":0,"ondisk_log_size":9155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37498880,"num_objects":9155,"num_object_clones":0,"num_object_copies":18310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9155,"num_write_kb":36620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9125","reported_seq":9344,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:22.956200+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:22.956200+0000","last_peered":"2024-09-16T04:52:22.956200+0000","last_clean":"2024-09-16T04:52:22.956200+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:22.956200+0000","last_undegraded":"2024-09-16T04:52:22.956200+0000","last_fullsized":"2024-09-16T04:52:22.956200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9125,"log_dups_size":0,"ondisk_log_size":9125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37253120,"num_objects":9095,"num_object_clones":0,"num_object_copies":18190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6157,"num_write_kb":24628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9163","reported_seq":9295,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:23.032560+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:23.032560+0000","last_peered":"2024-09-16T04:52:23.032560+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:23.032560+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9163,"log_dups_size":0,"ondisk_log_size":9163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37511168,"num_objects":9158,"num_object_clones":0,"num_object_copies":18316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6156,"num_write_kb":24624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5702},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9009","reported_seq":9157,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:23.007868+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:23.007868+0000","last_peered":"2024-09-16T04:52:23.007868+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:23.007868+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9009,"log_dups_size":0,"ondisk_log_size":9009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36876288,"num_objects":9003,"num_object_clones":0,"num_object_copies":18006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5664,"num_write_kb":22656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5604},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9052","reported_seq":9267,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:23.064981+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:23.064981+0000","last_peered":"2024-09-16T04:52:23.064981+0000","last_clean":"2024-09-16T04:52:23.064981+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:23.064981+0000","last_undegraded":"2024-09-16T04:52:23.064981+0000","last_fullsized":"2024-09-16T04:52:23.064981+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9052,"log_dups_size":0,"ondisk_log_size":9052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37044224,"num_objects":9044,"num_object_clones":0,"num_object_copies":18088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6122,"num_write_kb":24488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9108","reported_seq":9336,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:22.180475+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:22.180475+0000","last_peered":"2024-09-16T04:52:22.180475+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:22.180475+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9108,"log_dups_size":0,"ondisk_log_size":9108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37171200,"num_objects":9075,"num_object_clones":0,"num_object_copies":18150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5859,"num_write_kb":23436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5754}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'9138","reported_seq":9268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:23.011108+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:23.011108+0000","last_peered":"2024-09-16T04:52:23.011108+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:23.011108+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":9138,"log_dups_size":0,"ondisk_log_size":9138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37396480,"num_objects":9130,"num_object_clones":0,"num_object_copies":18260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5793,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5665},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'8973","reported_seq":9215,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:22.204848+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:22.204848+0000","last_peered":"2024-09-16T04:52:22.204848+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:22.204848+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":8973,"log_dups_size":0,"ondisk_log_size":8973,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36627666,"num_objects":8944,"num_object_clones":0,"num_object_copies":17888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8945,"num_write_kb":35771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5648}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9014","reported_seq":15870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:22.004210+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:22.004210+0000","last_peered":"2024-09-16T04:52:22.004210+0000","last_clean":"2024-09-16T04:52:22.004210+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:22.004210+0000","last_undegraded":"2024-09-16T04:52:22.004210+0000","last_fullsized":"2024-09-16T04:52:22.004210+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9014,"log_dups_size":0,"ondisk_log_size":9014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36896768,"num_objects":9008,"num_object_clones":0,"num_object_copies":18016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5737,"num_write_kb":22948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'8983","reported_seq":9079,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:22.224355+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:22.224355+0000","last_peered":"2024-09-16T04:52:22.224355+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:22.224355+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":8983,"log_dups_size":0,"ondisk_log_size":8983,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36786793,"num_objects":8982,"num_object_clones":0,"num_object_copies":17964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8982,"num_write_kb":35925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5575}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9280","reported_seq":9383,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:22.170871+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:22.170871+0000","last_peered":"2024-09-16T04:52:22.170871+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:22.170871+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9280,"log_dups_size":0,"ondisk_log_size":9280,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37979963,"num_objects":9275,"num_object_clones":0,"num_object_copies":18550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9277,"num_write_kb":37092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":5761}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9145","reported_seq":11031,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:23.093932+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:23.093932+0000","last_peered":"2024-09-16T04:52:23.093932+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:23.093932+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9145,"log_dups_size":0,"ondisk_log_size":9145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37441536,"num_objects":9141,"num_object_clones":0,"num_object_copies":18282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2446,"num_objects_unfound":0,"num_objects_dirty":9141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5899,"num_write_kb":23596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":877,"num_bytes_recovered":3592192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6693},{"shards":"0,2","objects":2446}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9235","reported_seq":9375,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:23.077758+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:23.077758+0000","last_peered":"2024-09-16T04:52:23.077758+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:23.077758+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9235,"log_dups_size":0,"ondisk_log_size":9235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37778025,"num_objects":9224,"num_object_clones":0,"num_object_copies":18448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6004,"num_write_kb":24016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5700},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":594649311,"num_objects":145184,"num_object_clones":0,"num_object_copies":290368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2446,"num_objects_unfound":0,"num_objects_dirty":145184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110418,"num_write_kb":441644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4232,"num_bytes_recovered":17301504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1212129280,"data_stored":1212049263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145340,"ondisk_log_size":145340,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":714532,"kb_used_data":444052,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93657308,"statfs":{"total":96636764160,"available":95905083392,"internally_reserved":0,"allocated":454709248,"data_stored":454440265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":276950697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,1,3,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593763,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532412,"kb_used_data":293868,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93839428,"statfs":{"total":96636764160,"available":96091574272,"internally_reserved":0,"allocated":300920832,"data_stored":300582269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729108,"kb_used_data":444804,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93642732,"statfs":{"total":96636764160,"available":95890157568,"internally_reserved":0,"allocated":455479296,"data_stored":455210313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":455503872,"data_stored":455500393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301830144,"data_stored":301757085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454795264,"data_stored":454791785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:27.186 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:52:27.186 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'8948", 'reported_seq': 9027, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:22.968146+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:52:22.968146+0000', 'last_peered': '2024-09-16T04:52:22.968146+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:52:22.968146+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 8948, 'log_dups_size': 0, 'ondisk_log_size': 8948, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36651008, 'num_objects': 8948, 'num_object_clones': 0, 'num_object_copies': 17896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8948, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8948, 'num_write_kb': 35792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5607}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:27.186 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:52:27.186 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'8963", 'reported_seq': 9101, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:22.137071+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:52:22.137071+0000', 'last_peered': '2024-09-16T04:52:22.137071+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:52:22.137071+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 8963, 'log_dups_size': 0, 'ondisk_log_size': 8963, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36696064, 'num_objects': 8959, 'num_object_clones': 0, 'num_object_copies': 17918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8959, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5702, 'num_write_kb': 22808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 5640}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:27.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:52:27.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'9163", 'reported_seq': 9295, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:23.032560+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:52:23.032560+0000', 'last_peered': '2024-09-16T04:52:23.032560+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:52:23.032560+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 9163, 'log_dups_size': 0, 'ondisk_log_size': 9163, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37511168, 'num_objects': 9158, 'num_object_clones': 0, 'num_object_copies': 18316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6156, 'num_write_kb': 24624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5702}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:27.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:52:27.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'9009", 'reported_seq': 9157, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:23.007868+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:52:23.007868+0000', 'last_peered': '2024-09-16T04:52:23.007868+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:52:23.007868+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 9009, 'log_dups_size': 0, 'ondisk_log_size': 9009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36876288, 'num_objects': 9003, 'num_object_clones': 0, 'num_object_copies': 18006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9003, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5664, 'num_write_kb': 22656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5604}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:27.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:52:27.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'9108", 'reported_seq': 9336, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:22.180475+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:52:22.180475+0000', 'last_peered': '2024-09-16T04:52:22.180475+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:52:22.180475+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 9108, 'log_dups_size': 0, 'ondisk_log_size': 9108, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37171200, 'num_objects': 9075, 'num_object_clones': 0, 'num_object_copies': 18150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9075, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5859, 'num_write_kb': 23436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 5754}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:27.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:52:27.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'9138", 'reported_seq': 9268, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:23.011108+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:52:23.011108+0000', 'last_peered': '2024-09-16T04:52:23.011108+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:52:23.011108+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 9138, 'log_dups_size': 0, 'ondisk_log_size': 9138, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37396480, 'num_objects': 9130, 'num_object_clones': 0, 'num_object_copies': 18260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5793, 'num_write_kb': 23172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5665}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:27.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:52:27.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'8973", 'reported_seq': 9215, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:22.204848+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:52:22.204848+0000', 'last_peered': '2024-09-16T04:52:22.204848+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:52:22.204848+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 8973, 'log_dups_size': 0, 'ondisk_log_size': 8973, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36627666, 'num_objects': 8944, 'num_object_clones': 0, 'num_object_copies': 17888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8944, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8945, 'num_write_kb': 35771, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 5648}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:27.187 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:52:27.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'8983", 'reported_seq': 9079, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:22.224355+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:52:22.224355+0000', 'last_peered': '2024-09-16T04:52:22.224355+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:52:22.224355+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 8983, 'log_dups_size': 0, 'ondisk_log_size': 8983, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36786793, 'num_objects': 8982, 'num_object_clones': 0, 'num_object_copies': 17964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8982, 'num_write_kb': 35925, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 5575}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:27.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:52:27.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'9280", 'reported_seq': 9383, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:22.170871+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:52:22.170871+0000', 'last_peered': '2024-09-16T04:52:22.170871+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:52:22.170871+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 9280, 'log_dups_size': 0, 'ondisk_log_size': 9280, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37979963, 'num_objects': 9275, 'num_object_clones': 0, 'num_object_copies': 18550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9275, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9277, 'num_write_kb': 37092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 5761}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:27.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:52:27.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'9145", 'reported_seq': 11031, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:23.093932+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:52:23.093932+0000', 'last_peered': '2024-09-16T04:52:23.093932+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:52:23.093932+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 9145, 'log_dups_size': 0, 'ondisk_log_size': 9145, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37441536, 'num_objects': 9141, 'num_object_clones': 0, 'num_object_copies': 18282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2446, 'num_objects_unfound': 0, 'num_objects_dirty': 9141, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5899, 'num_write_kb': 23596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 877, 'num_bytes_recovered': 3592192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6693}, {'shards': '0,2', 'objects': 2446}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:27.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:52:27.188 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'9235", 'reported_seq': 9375, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:23.077758+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:52:23.077758+0000', 'last_peered': '2024-09-16T04:52:23.077758+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:52:23.077758+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 9235, 'log_dups_size': 0, 'ondisk_log_size': 9235, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37778025, 'num_objects': 9224, 'num_object_clones': 0, 'num_object_copies': 18448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6004, 'num_write_kb': 24016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5700}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:27.188 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:52:28.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 211 20 147742 147722 2.73367 1.74609 0.03381 0.0285695 2024-09-16T04:52:28.175 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:28.175 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/290372 objects degraded (11.730%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":295,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":145186,"data_bytes":594649330,"bytes_used":2023477248,"bytes_avail":287886815232,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":290372,"degraded_ratio":0.1173046988001598,"misplaced_objects":2446,"misplaced_total":290372,"misplaced_ratio":0.0084236772140564523,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":59006,"recovering_keys_per_sec":0,"num_objects_recovered":173,"num_bytes_recovered":708608,"num_keys_recovered":0,"write_bytes_sec":1573052,"read_op_per_sec":0,"write_op_per_sec":384},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:52:28.217 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:52:28.217 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:29.109 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:29.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 212 20 148175 148155 2.72875 1.69141 0.0139584 0.0286214 2024-09-16T04:52:29.178 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-16T04:52:28.358241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604324082,"num_objects":147548,"num_object_clones":0,"num_object_copies":295096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2351,"num_objects_unfound":0,"num_objects_dirty":147548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112782,"num_write_kb":451094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4328,"num_bytes_recovered":17690643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147705,"ondisk_log_size":147705,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2014128,"kb_used_data":1194432,"kb_used_omap":12,"kb_used_meta":819635,"kb_avail":281101392,"statfs":{"total":289910292480,"available":287847825408,"internally_reserved":0,"allocated":1223098368,"data_stored":1222221839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":839306241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,9,4,8,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28565504,"num_objects":6974,"num_object_clones":0,"num_object_copies":13948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-268,"num_objects_unfound":0,"num_objects_dirty":6974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6974,"num_write_kb":27896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":1097728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005362"},"pg_stats":[{"pgid":"2.f","version":"47'9077","reported_seq":9156,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.120961+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:28.120961+0000","last_peered":"2024-09-16T04:52:28.120961+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:28.120961+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9077,"log_dups_size":0,"ondisk_log_size":9077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37179392,"num_objects":9077,"num_object_clones":0,"num_object_copies":18154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9077,"num_write_kb":36308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5736},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9193","reported_seq":9309,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:27.168871+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:27.168871+0000","last_peered":"2024-09-16T04:52:27.168871+0000","last_clean":"2024-09-16T04:52:27.168871+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:27.168871+0000","last_undegraded":"2024-09-16T04:52:27.168871+0000","last_fullsized":"2024-09-16T04:52:27.168871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9193,"log_dups_size":0,"ondisk_log_size":9193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629952,"num_objects":9187,"num_object_clones":0,"num_object_copies":18374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6162,"num_write_kb":24648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9098","reported_seq":9236,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:27.236688+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:27.236688+0000","last_peered":"2024-09-16T04:52:27.236688+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:27.236688+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9098,"log_dups_size":0,"ondisk_log_size":9098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37249024,"num_objects":9094,"num_object_clones":0,"num_object_copies":18188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5837,"num_write_kb":23348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5775}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9302","reported_seq":9372,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:27.149038+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:27.149038+0000","last_peered":"2024-09-16T04:52:27.149038+0000","last_clean":"2024-09-16T04:52:27.149038+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:27.149038+0000","last_undegraded":"2024-09-16T04:52:27.149038+0000","last_fullsized":"2024-09-16T04:52:27.149038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9302,"log_dups_size":0,"ondisk_log_size":9302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38100992,"num_objects":9302,"num_object_clones":0,"num_object_copies":18604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9302,"num_write_kb":37208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9268","reported_seq":9487,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:28.143189+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:28.143189+0000","last_peered":"2024-09-16T04:52:28.143189+0000","last_clean":"2024-09-16T04:52:28.143189+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:28.143189+0000","last_undegraded":"2024-09-16T04:52:28.143189+0000","last_fullsized":"2024-09-16T04:52:28.143189+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9268,"log_dups_size":0,"ondisk_log_size":9268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37838848,"num_objects":9238,"num_object_clones":0,"num_object_copies":18476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6300,"num_write_kb":25200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9327","reported_seq":9460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.104675+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:28.104675+0000","last_peered":"2024-09-16T04:52:28.104675+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:28.104675+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9327,"log_dups_size":0,"ondisk_log_size":9327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38182912,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6320,"num_write_kb":25280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5866},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9153","reported_seq":9301,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:27.963986+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:27.963986+0000","last_peered":"2024-09-16T04:52:27.963986+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:27.963986+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9153,"log_dups_size":0,"ondisk_log_size":9153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37466112,"num_objects":9147,"num_object_clones":0,"num_object_copies":18294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5808,"num_write_kb":23232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5748},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9224","reported_seq":9440,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:28.073309+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:28.073309+0000","last_peered":"2024-09-16T04:52:28.073309+0000","last_clean":"2024-09-16T04:52:28.073309+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:28.073309+0000","last_undegraded":"2024-09-16T04:52:28.073309+0000","last_fullsized":"2024-09-16T04:52:28.073309+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9224,"log_dups_size":0,"ondisk_log_size":9224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":9216,"num_object_clones":0,"num_object_copies":18432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6294,"num_write_kb":25176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9244","reported_seq":9472,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:27.232834+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:27.232834+0000","last_peered":"2024-09-16T04:52:27.232834+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:27.232834+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9244,"log_dups_size":0,"ondisk_log_size":9244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37728256,"num_objects":9211,"num_object_clones":0,"num_object_copies":18422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5995,"num_write_kb":23980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":5890}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'9294","reported_seq":9422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.144718+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:28.144718+0000","last_peered":"2024-09-16T04:52:28.144718+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:28.144718+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":9294,"log_dups_size":0,"ondisk_log_size":9294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38035456,"num_objects":9286,"num_object_clones":0,"num_object_copies":18572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5949,"num_write_kb":23796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5821},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'9137","reported_seq":9377,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:27.182099+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:27.182099+0000","last_peered":"2024-09-16T04:52:27.182099+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:27.182099+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":9137,"log_dups_size":0,"ondisk_log_size":9137,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37299410,"num_objects":9108,"num_object_clones":0,"num_object_copies":18216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9109,"num_write_kb":36427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5812}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9169","reported_seq":16025,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:27.221928+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:27.221928+0000","last_peered":"2024-09-16T04:52:27.221928+0000","last_clean":"2024-09-16T04:52:27.221928+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:27.221928+0000","last_undegraded":"2024-09-16T04:52:27.221928+0000","last_fullsized":"2024-09-16T04:52:27.221928+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9169,"log_dups_size":0,"ondisk_log_size":9169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37531648,"num_objects":9163,"num_object_clones":0,"num_object_copies":18326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5892,"num_write_kb":23568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9119","reported_seq":9217,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:27.218064+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:27.218064+0000","last_peered":"2024-09-16T04:52:27.218064+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:27.218064+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9119,"log_dups_size":0,"ondisk_log_size":9119,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343849,"num_objects":9118,"num_object_clones":0,"num_object_copies":18236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9118,"num_write_kb":36469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5711}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9413","reported_seq":9515,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:27.231027+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:27.231027+0000","last_peered":"2024-09-16T04:52:27.231027+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:27.231027+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9413,"log_dups_size":0,"ondisk_log_size":9413,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38524731,"num_objects":9408,"num_object_clones":0,"num_object_copies":18816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9410,"num_write_kb":37624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":5894}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9281","reported_seq":11357,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.141715+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:28.141715+0000","last_peered":"2024-09-16T04:52:28.141715+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:28.141715+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9281,"log_dups_size":0,"ondisk_log_size":9281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37998592,"num_objects":9277,"num_object_clones":0,"num_object_copies":18554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2351,"num_objects_unfound":0,"num_objects_dirty":9277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6035,"num_write_kb":24140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":972,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6924},{"shards":"0,2","objects":2351}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9403","reported_seq":9543,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.118532+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:28.118532+0000","last_peered":"2024-09-16T04:52:28.118532+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:28.118532+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9403,"log_dups_size":0,"ondisk_log_size":9403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38466153,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6172,"num_write_kb":24688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5868},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604324063,"num_objects":147546,"num_object_clones":0,"num_object_copies":295092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2351,"num_objects_unfound":0,"num_objects_dirty":147546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112780,"num_write_kb":451092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4327,"num_bytes_recovered":17690624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1224245248,"data_stored":1224165231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147702,"ondisk_log_size":147702,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":742536,"kb_used_data":449848,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93629304,"statfs":{"total":96636764160,"available":95876407296,"internally_reserved":0,"allocated":460644352,"data_stored":460375369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":299691689},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,7,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593763,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532412,"kb_used_data":293868,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93839428,"statfs":{"total":96636764160,"available":96091574272,"internally_reserved":0,"allocated":300920832,"data_stored":300582269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739180,"kb_used_data":450716,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93632660,"statfs":{"total":96636764160,"available":95879843840,"internally_reserved":0,"allocated":461533184,"data_stored":461264201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,7,2,0,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":461627392,"data_stored":461623913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301830144,"data_stored":301757085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":460787712,"data_stored":460784233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:30.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 213 20 148657 148637 2.72478 1.88281 0.0550613 0.0286649 2024-09-16T04:52:31.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 214 20 149118 149098 2.72046 1.80078 0.080601 0.0287104 2024-09-16T04:52:32.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 215 20 149589 149569 2.71637 1.83984 0.0304055 0.0287532 2024-09-16T04:52:32.181 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:33.043 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-16T04:52:32.359257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":609161458,"num_objects":148729,"num_object_clones":0,"num_object_copies":297458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2351,"num_objects_unfound":0,"num_objects_dirty":148729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113963,"num_write_kb":455818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4328,"num_bytes_recovered":17690643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148886,"ondisk_log_size":148886,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2006440,"kb_used_data":1207544,"kb_used_omap":12,"kb_used_meta":798835,"kb_avail":281109080,"statfs":{"total":289910292480,"available":287855697920,"internally_reserved":0,"allocated":1236525056,"data_stored":1235648527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":818007041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,9,4,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23814144,"num_objects":5814,"num_object_clones":0,"num_object_copies":11628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-188,"num_objects_unfound":0,"num_objects_dirty":5814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5814,"num_write_kb":23256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":770048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003848"},"pg_stats":[{"pgid":"2.f","version":"47'9077","reported_seq":9156,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.120961+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:28.120961+0000","last_peered":"2024-09-16T04:52:28.120961+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:28.120961+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9077,"log_dups_size":0,"ondisk_log_size":9077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37179392,"num_objects":9077,"num_object_clones":0,"num_object_copies":18154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9077,"num_write_kb":36308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5736},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9329","reported_seq":9445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:32.234413+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:32.234413+0000","last_peered":"2024-09-16T04:52:32.234413+0000","last_clean":"2024-09-16T04:52:32.234413+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:32.234413+0000","last_undegraded":"2024-09-16T04:52:32.234413+0000","last_fullsized":"2024-09-16T04:52:32.234413+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9329,"log_dups_size":0,"ondisk_log_size":9329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38187008,"num_objects":9323,"num_object_clones":0,"num_object_copies":18646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6298,"num_write_kb":25192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9252","reported_seq":9390,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.219710+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:32.219710+0000","last_peered":"2024-09-16T04:52:32.219710+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:32.219710+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9252,"log_dups_size":0,"ondisk_log_size":9252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37879808,"num_objects":9248,"num_object_clones":0,"num_object_copies":18496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5991,"num_write_kb":23964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5929}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9453","reported_seq":9523,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:32.301906+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:32.301906+0000","last_peered":"2024-09-16T04:52:32.301906+0000","last_clean":"2024-09-16T04:52:32.301906+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:32.301906+0000","last_undegraded":"2024-09-16T04:52:32.301906+0000","last_fullsized":"2024-09-16T04:52:32.301906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9453,"log_dups_size":0,"ondisk_log_size":9453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38719488,"num_objects":9453,"num_object_clones":0,"num_object_copies":18906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9453,"num_write_kb":37812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9268","reported_seq":9487,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:28.143189+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:28.143189+0000","last_peered":"2024-09-16T04:52:28.143189+0000","last_clean":"2024-09-16T04:52:28.143189+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:28.143189+0000","last_undegraded":"2024-09-16T04:52:28.143189+0000","last_fullsized":"2024-09-16T04:52:28.143189+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9268,"log_dups_size":0,"ondisk_log_size":9268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37838848,"num_objects":9238,"num_object_clones":0,"num_object_copies":18476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6300,"num_write_kb":25200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9327","reported_seq":9460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.104675+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:28.104675+0000","last_peered":"2024-09-16T04:52:28.104675+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:28.104675+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9327,"log_dups_size":0,"ondisk_log_size":9327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38182912,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6320,"num_write_kb":25280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5866},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9153","reported_seq":9301,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:27.963986+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:27.963986+0000","last_peered":"2024-09-16T04:52:27.963986+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:27.963986+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9153,"log_dups_size":0,"ondisk_log_size":9153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37466112,"num_objects":9147,"num_object_clones":0,"num_object_copies":18294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5808,"num_write_kb":23232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5748},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9224","reported_seq":9440,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:28.073309+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:28.073309+0000","last_peered":"2024-09-16T04:52:28.073309+0000","last_clean":"2024-09-16T04:52:28.073309+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:28.073309+0000","last_undegraded":"2024-09-16T04:52:28.073309+0000","last_fullsized":"2024-09-16T04:52:28.073309+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9224,"log_dups_size":0,"ondisk_log_size":9224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":9216,"num_object_clones":0,"num_object_copies":18432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6294,"num_write_kb":25176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9381","reported_seq":9609,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.186962+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:32.186962+0000","last_peered":"2024-09-16T04:52:32.186962+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:32.186962+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9381,"log_dups_size":0,"ondisk_log_size":9381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38289408,"num_objects":9348,"num_object_clones":0,"num_object_copies":18696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6132,"num_write_kb":24528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":6027}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'9294","reported_seq":9422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.144718+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:28.144718+0000","last_peered":"2024-09-16T04:52:28.144718+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:28.144718+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":9294,"log_dups_size":0,"ondisk_log_size":9294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38035456,"num_objects":9286,"num_object_clones":0,"num_object_copies":18572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5949,"num_write_kb":23796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5821},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'9297","reported_seq":9539,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.173027+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:32.173027+0000","last_peered":"2024-09-16T04:52:32.173027+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:32.173027+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":9297,"log_dups_size":0,"ondisk_log_size":9297,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37954770,"num_objects":9268,"num_object_clones":0,"num_object_copies":18536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9269,"num_write_kb":37067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9303","reported_seq":16159,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:32.267691+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:32.267691+0000","last_peered":"2024-09-16T04:52:32.267691+0000","last_clean":"2024-09-16T04:52:32.267691+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:32.267691+0000","last_undegraded":"2024-09-16T04:52:32.267691+0000","last_fullsized":"2024-09-16T04:52:32.267691+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9303,"log_dups_size":0,"ondisk_log_size":9303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38080512,"num_objects":9297,"num_object_clones":0,"num_object_copies":18594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6026,"num_write_kb":24104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9261","reported_seq":9359,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.248861+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:32.248861+0000","last_peered":"2024-09-16T04:52:32.248861+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:32.248861+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9261,"log_dups_size":0,"ondisk_log_size":9261,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37925481,"num_objects":9260,"num_object_clones":0,"num_object_copies":18520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9260,"num_write_kb":37037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5853}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9580","reported_seq":9682,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.287837+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:32.287837+0000","last_peered":"2024-09-16T04:52:32.287837+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:32.287837+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9580,"log_dups_size":0,"ondisk_log_size":9580,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39208763,"num_objects":9575,"num_object_clones":0,"num_object_copies":19150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9577,"num_write_kb":38292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":6061}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9281","reported_seq":11357,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.141715+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:28.141715+0000","last_peered":"2024-09-16T04:52:28.141715+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:28.141715+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9281,"log_dups_size":0,"ondisk_log_size":9281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37998592,"num_objects":9277,"num_object_clones":0,"num_object_copies":18554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2351,"num_objects_unfound":0,"num_objects_dirty":9277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6035,"num_write_kb":24140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":972,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6924},{"shards":"0,2","objects":2351}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9403","reported_seq":9543,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.118532+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:28.118532+0000","last_peered":"2024-09-16T04:52:28.118532+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:28.118532+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9403,"log_dups_size":0,"ondisk_log_size":9403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38466153,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6172,"num_write_kb":24688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5868},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":609161439,"num_objects":148727,"num_object_clones":0,"num_object_copies":297454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2351,"num_objects_unfound":0,"num_objects_dirty":148727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113961,"num_write_kb":455816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4327,"num_bytes_recovered":17690624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1237798912,"data_stored":1237718895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148883,"ondisk_log_size":148883,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735940,"kb_used_data":455732,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93635900,"statfs":{"total":96636764160,"available":95883161600,"internally_reserved":0,"allocated":466669568,"data_stored":466400585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":286912169},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593764,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531320,"kb_used_data":301096,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93840520,"statfs":{"total":96636764160,"available":96092692480,"internally_reserved":0,"allocated":308322304,"data_stored":307983741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739180,"kb_used_data":450716,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93632660,"statfs":{"total":96636764160,"available":95879843840,"internally_reserved":0,"allocated":461533184,"data_stored":461264201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,7,2,0,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":461627392,"data_stored":461623913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309403648,"data_stored":309330589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466767872,"data_stored":466764393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:33.044 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:33.132 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 216 20 150027 150007 2.71171 1.71094 0.0588939 0.0288032 2024-09-16T04:52:33.937 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:33.938 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:33.991 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-16T04:52:32.359257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":609161458,"num_objects":148729,"num_object_clones":0,"num_object_copies":297458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2351,"num_objects_unfound":0,"num_objects_dirty":148729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113963,"num_write_kb":455818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4328,"num_bytes_recovered":17690643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148886,"ondisk_log_size":148886,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2006440,"kb_used_data":1207544,"kb_used_omap":12,"kb_used_meta":798835,"kb_avail":281109080,"statfs":{"total":289910292480,"available":287855697920,"internally_reserved":0,"allocated":1236525056,"data_stored":1235648527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":818007041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,9,4,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23814144,"num_objects":5814,"num_object_clones":0,"num_object_copies":11628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-188,"num_objects_unfound":0,"num_objects_dirty":5814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5814,"num_write_kb":23256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":770048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003848"},"pg_stats":[{"pgid":"2.f","version":"47'9077","reported_seq":9156,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.120961+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:28.120961+0000","last_peered":"2024-09-16T04:52:28.120961+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:28.120961+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9077,"log_dups_size":0,"ondisk_log_size":9077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37179392,"num_objects":9077,"num_object_clones":0,"num_object_copies":18154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9077,"num_write_kb":36308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5736},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9329","reported_seq":9445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:32.234413+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:32.234413+0000","last_peered":"2024-09-16T04:52:32.234413+0000","last_clean":"2024-09-16T04:52:32.234413+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:32.234413+0000","last_undegraded":"2024-09-16T04:52:32.234413+0000","last_fullsized":"2024-09-16T04:52:32.234413+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9329,"log_dups_size":0,"ondisk_log_size":9329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38187008,"num_objects":9323,"num_object_clones":0,"num_object_copies":18646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6298,"num_write_kb":25192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9252","reported_seq":9390,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.219710+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:32.219710+0000","last_peered":"2024-09-16T04:52:32.219710+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:32.219710+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9252,"log_dups_size":0,"ondisk_log_size":9252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37879808,"num_objects":9248,"num_object_clones":0,"num_object_copies":18496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5991,"num_write_kb":23964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5929}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9453","reported_seq":9523,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:32.301906+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:32.301906+0000","last_peered":"2024-09-16T04:52:32.301906+0000","last_clean":"2024-09-16T04:52:32.301906+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:32.301906+0000","last_undegraded":"2024-09-16T04:52:32.301906+0000","last_fullsized":"2024-09-16T04:52:32.301906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9453,"log_dups_size":0,"ondisk_log_size":9453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38719488,"num_objects":9453,"num_object_clones":0,"num_object_copies":18906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9453,"num_write_kb":37812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9268","reported_seq":9487,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:28.143189+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:28.143189+0000","last_peered":"2024-09-16T04:52:28.143189+0000","last_clean":"2024-09-16T04:52:28.143189+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:28.143189+0000","last_undegraded":"2024-09-16T04:52:28.143189+0000","last_fullsized":"2024-09-16T04:52:28.143189+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9268,"log_dups_size":0,"ondisk_log_size":9268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37838848,"num_objects":9238,"num_object_clones":0,"num_object_copies":18476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6300,"num_write_kb":25200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9327","reported_seq":9460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.104675+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:28.104675+0000","last_peered":"2024-09-16T04:52:28.104675+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:28.104675+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9327,"log_dups_size":0,"ondisk_log_size":9327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38182912,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6320,"num_write_kb":25280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5866},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9153","reported_seq":9301,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:27.963986+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:27.963986+0000","last_peered":"2024-09-16T04:52:27.963986+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:27.963986+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9153,"log_dups_size":0,"ondisk_log_size":9153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37466112,"num_objects":9147,"num_object_clones":0,"num_object_copies":18294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5808,"num_write_kb":23232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5748},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9224","reported_seq":9440,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:28.073309+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:28.073309+0000","last_peered":"2024-09-16T04:52:28.073309+0000","last_clean":"2024-09-16T04:52:28.073309+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:28.073309+0000","last_undegraded":"2024-09-16T04:52:28.073309+0000","last_fullsized":"2024-09-16T04:52:28.073309+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9224,"log_dups_size":0,"ondisk_log_size":9224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":9216,"num_object_clones":0,"num_object_copies":18432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6294,"num_write_kb":25176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9381","reported_seq":9609,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.186962+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:32.186962+0000","last_peered":"2024-09-16T04:52:32.186962+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:32.186962+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9381,"log_dups_size":0,"ondisk_log_size":9381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38289408,"num_objects":9348,"num_object_clones":0,"num_object_copies":18696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6132,"num_write_kb":24528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":6027}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'9294","reported_seq":9422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.144718+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:28.144718+0000","last_peered":"2024-09-16T04:52:28.144718+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:28.144718+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":9294,"log_dups_size":0,"ondisk_log_size":9294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38035456,"num_objects":9286,"num_object_clones":0,"num_object_copies":18572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5949,"num_write_kb":23796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5821},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154253+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154253+0000","last_peered":"2024-09-16T04:51:33.154253+0000","last_clean":"2024-09-16T04:51:33.154253+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154253+0000","last_undegraded":"2024-09-16T04:51:33.154253+0000","last_fullsized":"2024-09-16T04:51:33.154253+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'9297","reported_seq":9539,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.173027+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:32.173027+0000","last_peered":"2024-09-16T04:52:32.173027+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:32.173027+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":9297,"log_dups_size":0,"ondisk_log_size":9297,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37954770,"num_objects":9268,"num_object_clones":0,"num_object_copies":18536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9269,"num_write_kb":37067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9303","reported_seq":16159,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:32.267691+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:32.267691+0000","last_peered":"2024-09-16T04:52:32.267691+0000","last_clean":"2024-09-16T04:52:32.267691+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:32.267691+0000","last_undegraded":"2024-09-16T04:52:32.267691+0000","last_fullsized":"2024-09-16T04:52:32.267691+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9303,"log_dups_size":0,"ondisk_log_size":9303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38080512,"num_objects":9297,"num_object_clones":0,"num_object_copies":18594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6026,"num_write_kb":24104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9261","reported_seq":9359,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.248861+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:32.248861+0000","last_peered":"2024-09-16T04:52:32.248861+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:32.248861+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9261,"log_dups_size":0,"ondisk_log_size":9261,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37925481,"num_objects":9260,"num_object_clones":0,"num_object_copies":18520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9260,"num_write_kb":37037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5853}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9580","reported_seq":9682,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.287837+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:32.287837+0000","last_peered":"2024-09-16T04:52:32.287837+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:32.287837+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9580,"log_dups_size":0,"ondisk_log_size":9580,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39208763,"num_objects":9575,"num_object_clones":0,"num_object_copies":19150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9577,"num_write_kb":38292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":6061}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9281","reported_seq":11357,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.141715+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:28.141715+0000","last_peered":"2024-09-16T04:52:28.141715+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:28.141715+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9281,"log_dups_size":0,"ondisk_log_size":9281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37998592,"num_objects":9277,"num_object_clones":0,"num_object_copies":18554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2351,"num_objects_unfound":0,"num_objects_dirty":9277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6035,"num_write_kb":24140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":972,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6924},{"shards":"0,2","objects":2351}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9403","reported_seq":9543,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:28.118532+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:28.118532+0000","last_peered":"2024-09-16T04:52:28.118532+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:28.118532+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9403,"log_dups_size":0,"ondisk_log_size":9403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38466153,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6172,"num_write_kb":24688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5868},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":609161439,"num_objects":148727,"num_object_clones":0,"num_object_copies":297454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2351,"num_objects_unfound":0,"num_objects_dirty":148727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113961,"num_write_kb":455816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4327,"num_bytes_recovered":17690624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1237798912,"data_stored":1237718895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148883,"ondisk_log_size":148883,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735940,"kb_used_data":455732,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93635900,"statfs":{"total":96636764160,"available":95883161600,"internally_reserved":0,"allocated":466669568,"data_stored":466400585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":286912169},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593764,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531320,"kb_used_data":301096,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93840520,"statfs":{"total":96636764160,"available":96092692480,"internally_reserved":0,"allocated":308322304,"data_stored":307983741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739180,"kb_used_data":450716,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93632660,"statfs":{"total":96636764160,"available":95879843840,"internally_reserved":0,"allocated":461533184,"data_stored":461264201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,7,2,0,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":461627392,"data_stored":461623913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309403648,"data_stored":309330589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466767872,"data_stored":466764393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:33.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:52:33.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'9077", 'reported_seq': 9156, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:28.120961+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:52:28.120961+0000', 'last_peered': '2024-09-16T04:52:28.120961+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:52:28.120961+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 9077, 'log_dups_size': 0, 'ondisk_log_size': 9077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37179392, 'num_objects': 9077, 'num_object_clones': 0, 'num_object_copies': 18154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9077, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9077, 'num_write_kb': 36308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5736}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:33.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:52:33.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'9252", 'reported_seq': 9390, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:32.219710+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:52:32.219710+0000', 'last_peered': '2024-09-16T04:52:32.219710+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:52:32.219710+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 9252, 'log_dups_size': 0, 'ondisk_log_size': 9252, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37879808, 'num_objects': 9248, 'num_object_clones': 0, 'num_object_copies': 18496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9248, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5991, 'num_write_kb': 23964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 5929}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:33.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:52:33.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'9327", 'reported_seq': 9460, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:28.104675+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:52:28.104675+0000', 'last_peered': '2024-09-16T04:52:28.104675+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:52:28.104675+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 9327, 'log_dups_size': 0, 'ondisk_log_size': 9327, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38182912, 'num_objects': 9322, 'num_object_clones': 0, 'num_object_copies': 18644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9322, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6320, 'num_write_kb': 25280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5866}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:33.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:52:33.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'9153", 'reported_seq': 9301, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:27.963986+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:52:27.963986+0000', 'last_peered': '2024-09-16T04:52:27.963986+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:52:27.963986+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 9153, 'log_dups_size': 0, 'ondisk_log_size': 9153, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37466112, 'num_objects': 9147, 'num_object_clones': 0, 'num_object_copies': 18294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9147, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5808, 'num_write_kb': 23232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5748}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:33.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:52:33.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'9381", 'reported_seq': 9609, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:32.186962+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:52:32.186962+0000', 'last_peered': '2024-09-16T04:52:32.186962+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:52:32.186962+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 9381, 'log_dups_size': 0, 'ondisk_log_size': 9381, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38289408, 'num_objects': 9348, 'num_object_clones': 0, 'num_object_copies': 18696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9348, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6132, 'num_write_kb': 24528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 6027}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:33.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:52:33.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'9294", 'reported_seq': 9422, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:28.144718+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:52:28.144718+0000', 'last_peered': '2024-09-16T04:52:28.144718+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:52:28.144718+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 9294, 'log_dups_size': 0, 'ondisk_log_size': 9294, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38035456, 'num_objects': 9286, 'num_object_clones': 0, 'num_object_copies': 18572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9286, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5949, 'num_write_kb': 23796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5821}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:33.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:52:33.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'9297", 'reported_seq': 9539, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:32.173027+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:52:32.173027+0000', 'last_peered': '2024-09-16T04:52:32.173027+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:52:32.173027+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 9297, 'log_dups_size': 0, 'ondisk_log_size': 9297, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37954770, 'num_objects': 9268, 'num_object_clones': 0, 'num_object_copies': 18536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9268, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9269, 'num_write_kb': 37067, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 5972}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:33.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:52:33.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'9261", 'reported_seq': 9359, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:32.248861+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:52:32.248861+0000', 'last_peered': '2024-09-16T04:52:32.248861+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:52:32.248861+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 9261, 'log_dups_size': 0, 'ondisk_log_size': 9261, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37925481, 'num_objects': 9260, 'num_object_clones': 0, 'num_object_copies': 18520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9260, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9260, 'num_write_kb': 37037, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 5853}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:33.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:52:33.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'9580", 'reported_seq': 9682, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:32.287837+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:52:32.287837+0000', 'last_peered': '2024-09-16T04:52:32.287837+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:52:32.287837+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 9580, 'log_dups_size': 0, 'ondisk_log_size': 9580, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39208763, 'num_objects': 9575, 'num_object_clones': 0, 'num_object_copies': 19150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9575, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9577, 'num_write_kb': 38292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 6061}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:33.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:52:33.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'9281", 'reported_seq': 11357, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:28.141715+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:52:28.141715+0000', 'last_peered': '2024-09-16T04:52:28.141715+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:52:28.141715+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 9281, 'log_dups_size': 0, 'ondisk_log_size': 9281, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37998592, 'num_objects': 9277, 'num_object_clones': 0, 'num_object_copies': 18554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2351, 'num_objects_unfound': 0, 'num_objects_dirty': 9277, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6035, 'num_write_kb': 24140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 972, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6924}, {'shards': '0,2', 'objects': 2351}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:33.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:52:33.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'9403", 'reported_seq': 9543, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:28.118532+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:52:28.118532+0000', 'last_peered': '2024-09-16T04:52:28.118532+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:52:28.118532+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 9403, 'log_dups_size': 0, 'ondisk_log_size': 9403, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38466153, 'num_objects': 9392, 'num_object_clones': 0, 'num_object_copies': 18784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9392, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6172, 'num_write_kb': 24688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5868}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:33.995 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:52:34.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 217 20 150458 150438 2.70694 1.68359 0.0225928 0.0288515 2024-09-16T04:52:35.091 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:35.092 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/299688 objects degraded (11.366%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":302,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":149844,"data_bytes":613728498,"bytes_used":2064498688,"bytes_avail":287845793792,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":299688,"degraded_ratio":0.11365820453271402,"misplaced_objects":2243,"misplaced_total":299688,"misplaced_ratio":0.0074844504951816545,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":101009,"recovering_keys_per_sec":0,"num_objects_recovered":296,"num_bytes_recovered":1212416,"num_keys_recovered":0,"write_bytes_sec":1985389,"read_op_per_sec":0,"write_op_per_sec":484},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:52:35.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:52:35.126 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:35.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 218 20 150899 150879 2.70243 1.72266 0.0384216 0.0289012 2024-09-16T04:52:35.996 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:35.997 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:36.046 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-16T04:52:34.359746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613728498,"num_objects":149844,"num_object_clones":0,"num_object_copies":299688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2243,"num_objects_unfound":0,"num_objects_dirty":149844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115078,"num_write_kb":460278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4436,"num_bytes_recovered":18133011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150001,"ondisk_log_size":150001,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2016112,"kb_used_data":1213056,"kb_used_omap":12,"kb_used_meta":802995,"kb_avail":281099408,"statfs":{"total":289910292480,"available":287845793792,"internally_reserved":0,"allocated":1242169344,"data_stored":1241292815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":822266881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,4,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23830528,"num_objects":5818,"num_object_clones":0,"num_object_copies":11636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-296,"num_objects_unfound":0,"num_objects_dirty":5818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5818,"num_write_kb":23272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":1212416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002949"},"pg_stats":[{"pgid":"2.f","version":"47'9215","reported_seq":9294,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:33.113658+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:33.113658+0000","last_peered":"2024-09-16T04:52:33.113658+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:33.113658+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9215,"log_dups_size":0,"ondisk_log_size":9215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37744640,"num_objects":9215,"num_object_clones":0,"num_object_copies":18430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9215,"num_write_kb":36860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5874},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9329","reported_seq":9445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:32.234413+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:32.234413+0000","last_peered":"2024-09-16T04:52:32.234413+0000","last_clean":"2024-09-16T04:52:32.234413+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:32.234413+0000","last_undegraded":"2024-09-16T04:52:32.234413+0000","last_fullsized":"2024-09-16T04:52:32.234413+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9329,"log_dups_size":0,"ondisk_log_size":9329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38187008,"num_objects":9323,"num_object_clones":0,"num_object_copies":18646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6298,"num_write_kb":25192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9252","reported_seq":9390,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.219710+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:32.219710+0000","last_peered":"2024-09-16T04:52:32.219710+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:32.219710+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9252,"log_dups_size":0,"ondisk_log_size":9252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37879808,"num_objects":9248,"num_object_clones":0,"num_object_copies":18496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5991,"num_write_kb":23964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":5929}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9453","reported_seq":9523,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:32.301906+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:32.301906+0000","last_peered":"2024-09-16T04:52:32.301906+0000","last_clean":"2024-09-16T04:52:32.301906+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:32.301906+0000","last_undegraded":"2024-09-16T04:52:32.301906+0000","last_fullsized":"2024-09-16T04:52:32.301906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9453,"log_dups_size":0,"ondisk_log_size":9453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38719488,"num_objects":9453,"num_object_clones":0,"num_object_copies":18906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9453,"num_write_kb":37812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9405","reported_seq":9624,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:33.088184+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:33.088184+0000","last_peered":"2024-09-16T04:52:33.088184+0000","last_clean":"2024-09-16T04:52:33.088184+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:33.088184+0000","last_undegraded":"2024-09-16T04:52:33.088184+0000","last_fullsized":"2024-09-16T04:52:33.088184+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9405,"log_dups_size":0,"ondisk_log_size":9405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38400000,"num_objects":9375,"num_object_clones":0,"num_object_copies":18750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6437,"num_write_kb":25748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9479","reported_seq":9612,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:33.108667+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:33.108667+0000","last_peered":"2024-09-16T04:52:33.108667+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:33.108667+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9479,"log_dups_size":0,"ondisk_log_size":9479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38805504,"num_objects":9474,"num_object_clones":0,"num_object_copies":18948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6472,"num_write_kb":25888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6018},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9303","reported_seq":9451,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:33.177232+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:33.177232+0000","last_peered":"2024-09-16T04:52:33.177232+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:33.177232+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9303,"log_dups_size":0,"ondisk_log_size":9303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38080512,"num_objects":9297,"num_object_clones":0,"num_object_copies":18594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5958,"num_write_kb":23832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5898},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9345","reported_seq":9561,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:33.079902+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:33.079902+0000","last_peered":"2024-09-16T04:52:33.079902+0000","last_clean":"2024-09-16T04:52:33.079902+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:33.079902+0000","last_undegraded":"2024-09-16T04:52:33.079902+0000","last_fullsized":"2024-09-16T04:52:33.079902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9345,"log_dups_size":0,"ondisk_log_size":9345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38244352,"num_objects":9337,"num_object_clones":0,"num_object_copies":18674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6415,"num_write_kb":25660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9381","reported_seq":9609,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.186962+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:32.186962+0000","last_peered":"2024-09-16T04:52:32.186962+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:32.186962+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9381,"log_dups_size":0,"ondisk_log_size":9381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38289408,"num_objects":9348,"num_object_clones":0,"num_object_copies":18696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6132,"num_write_kb":24528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":6027}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'9438","reported_seq":9568,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:33.090513+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:33.090513+0000","last_peered":"2024-09-16T04:52:33.090513+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:33.090513+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":9438,"log_dups_size":0,"ondisk_log_size":9438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38625280,"num_objects":9430,"num_object_clones":0,"num_object_copies":18860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6093,"num_write_kb":24372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5965},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'9297","reported_seq":9539,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.173027+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:32.173027+0000","last_peered":"2024-09-16T04:52:32.173027+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:32.173027+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":9297,"log_dups_size":0,"ondisk_log_size":9297,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37954770,"num_objects":9268,"num_object_clones":0,"num_object_copies":18536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9269,"num_write_kb":37067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":5972}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9303","reported_seq":16159,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:32.267691+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:32.267691+0000","last_peered":"2024-09-16T04:52:32.267691+0000","last_clean":"2024-09-16T04:52:32.267691+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:32.267691+0000","last_undegraded":"2024-09-16T04:52:32.267691+0000","last_fullsized":"2024-09-16T04:52:32.267691+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9303,"log_dups_size":0,"ondisk_log_size":9303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38080512,"num_objects":9297,"num_object_clones":0,"num_object_copies":18594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6026,"num_write_kb":24104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9261","reported_seq":9359,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.248861+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:32.248861+0000","last_peered":"2024-09-16T04:52:32.248861+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:32.248861+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9261,"log_dups_size":0,"ondisk_log_size":9261,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37925481,"num_objects":9260,"num_object_clones":0,"num_object_copies":18520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9260,"num_write_kb":37037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5853}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9580","reported_seq":9682,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:32.287837+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:32.287837+0000","last_peered":"2024-09-16T04:52:32.287837+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:32.287837+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9580,"log_dups_size":0,"ondisk_log_size":9580,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39208763,"num_objects":9575,"num_object_clones":0,"num_object_copies":19150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9577,"num_write_kb":38292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":6061}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9407","reported_seq":11695,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:33.183195+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:33.183195+0000","last_peered":"2024-09-16T04:52:33.183195+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:33.183195+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9407,"log_dups_size":0,"ondisk_log_size":9407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514688,"num_objects":9403,"num_object_clones":0,"num_object_copies":18806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2243,"num_objects_unfound":0,"num_objects_dirty":9403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6161,"num_write_kb":24644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1080,"num_bytes_recovered":4423680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7158},{"shards":"0,2","objects":2243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9550","reported_seq":9690,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:33.138066+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:33.138066+0000","last_peered":"2024-09-16T04:52:33.138066+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:33.138066+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9550,"log_dups_size":0,"ondisk_log_size":9550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39068265,"num_objects":9539,"num_object_clones":0,"num_object_copies":19078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6319,"num_write_kb":25276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6015},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":613728479,"num_objects":149842,"num_object_clones":0,"num_object_copies":299684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2243,"num_objects_unfound":0,"num_objects_dirty":149842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115076,"num_write_kb":460276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4435,"num_bytes_recovered":18132992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1243529216,"data_stored":1243449199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149998,"ondisk_log_size":149998,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735940,"kb_used_data":455732,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93635900,"statfs":{"total":96636764160,"available":95883161600,"internally_reserved":0,"allocated":466669568,"data_stored":466400585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":286912169},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593764,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531320,"kb_used_data":301096,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93840520,"statfs":{"total":96636764160,"available":96092692480,"internally_reserved":0,"allocated":308322304,"data_stored":307983741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":748852,"kb_used_data":456228,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93622988,"statfs":{"total":96636764160,"available":95869939712,"internally_reserved":0,"allocated":467177472,"data_stored":466908489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":467357696,"data_stored":467354217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309403648,"data_stored":309330589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466767872,"data_stored":466764393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:36.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 219 20 151345 151325 2.69804 1.74219 0.0229914 0.0289476 2024-09-16T04:52:37.135 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:2024-09-16T04:52:37.134628+0000 min lat: 0.00206892 max lat: 2.69414 avg lat: 0.0289876 2024-09-16T04:52:37.136 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T04:52:37.136 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 220 20 151833 151813 2.69445 1.90625 0.0230858 0.0289876 2024-09-16T04:52:38.136 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 221 20 152304 152284 2.69058 1.83984 0.00927747 0.0290285 2024-09-16T04:52:39.049 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:39.136 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 222 20 152774 152754 2.68673 1.83594 0.0927659 0.0290702 2024-09-16T04:52:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:40.008 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:40.049 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-16T04:52:38.360833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":623112434,"num_objects":152135,"num_object_clones":0,"num_object_copies":304270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2164,"num_objects_unfound":0,"num_objects_dirty":152135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117369,"num_write_kb":469442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4515,"num_bytes_recovered":18456595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152292,"ondisk_log_size":152292,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2040112,"kb_used_data":1232896,"kb_used_omap":12,"kb_used_meta":807155,"kb_avail":281075408,"statfs":{"total":289910292480,"available":287821217792,"internally_reserved":0,"allocated":1262485504,"data_stored":1261608975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":826526721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,4,1,13,15,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28463104,"num_objects":6949,"num_object_clones":0,"num_object_copies":13898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":6949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6949,"num_write_kb":27796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":1155072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003069"},"pg_stats":[{"pgid":"2.f","version":"47'9361","reported_seq":9440,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.184538+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:38.184538+0000","last_peered":"2024-09-16T04:52:38.184538+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:38.184538+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9361,"log_dups_size":0,"ondisk_log_size":9361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38342656,"num_objects":9361,"num_object_clones":0,"num_object_copies":18722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9361,"num_write_kb":37444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6020},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9446","reported_seq":9562,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:37.331426+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:37.331426+0000","last_peered":"2024-09-16T04:52:37.331426+0000","last_clean":"2024-09-16T04:52:37.331426+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:37.331426+0000","last_undegraded":"2024-09-16T04:52:37.331426+0000","last_fullsized":"2024-09-16T04:52:37.331426+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9446,"log_dups_size":0,"ondisk_log_size":9446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38666240,"num_objects":9440,"num_object_clones":0,"num_object_copies":18880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6415,"num_write_kb":25660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9398","reported_seq":9536,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:37.325270+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:37.325270+0000","last_peered":"2024-09-16T04:52:37.325270+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:37.325270+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9398,"log_dups_size":0,"ondisk_log_size":9398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38477824,"num_objects":9394,"num_object_clones":0,"num_object_copies":18788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6137,"num_write_kb":24548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":6075}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9591","reported_seq":9661,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:37.321170+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:37.321170+0000","last_peered":"2024-09-16T04:52:37.321170+0000","last_clean":"2024-09-16T04:52:37.321170+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:37.321170+0000","last_undegraded":"2024-09-16T04:52:37.321170+0000","last_fullsized":"2024-09-16T04:52:37.321170+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9591,"log_dups_size":0,"ondisk_log_size":9591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9591,"num_write_kb":38364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9551","reported_seq":9770,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:38.121744+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:38.121744+0000","last_peered":"2024-09-16T04:52:38.121744+0000","last_clean":"2024-09-16T04:52:38.121744+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:38.121744+0000","last_undegraded":"2024-09-16T04:52:38.121744+0000","last_fullsized":"2024-09-16T04:52:38.121744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9551,"log_dups_size":0,"ondisk_log_size":9551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38998016,"num_objects":9521,"num_object_clones":0,"num_object_copies":19042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6583,"num_write_kb":26332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9609","reported_seq":9742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.152929+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:38.152929+0000","last_peered":"2024-09-16T04:52:38.152929+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:38.152929+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9609,"log_dups_size":0,"ondisk_log_size":9609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6602,"num_write_kb":26408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6148},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9450","reported_seq":9597,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.172977+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:38.172977+0000","last_peered":"2024-09-16T04:52:38.172977+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:38.172977+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9450,"log_dups_size":0,"ondisk_log_size":9450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38682624,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6105,"num_write_kb":24420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6045},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9503","reported_seq":9718,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:38.173296+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:38.173296+0000","last_peered":"2024-09-16T04:52:38.173296+0000","last_clean":"2024-09-16T04:52:38.173296+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:38.173296+0000","last_undegraded":"2024-09-16T04:52:38.173296+0000","last_fullsized":"2024-09-16T04:52:38.173296+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9503,"log_dups_size":0,"ondisk_log_size":9503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38891520,"num_objects":9495,"num_object_clones":0,"num_object_copies":18990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6573,"num_write_kb":26292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9505","reported_seq":9733,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:37.243593+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:37.243593+0000","last_peered":"2024-09-16T04:52:37.243593+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:37.243593+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9505,"log_dups_size":0,"ondisk_log_size":9505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38797312,"num_objects":9472,"num_object_clones":0,"num_object_copies":18944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6256,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":6151}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'9595","reported_seq":9725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.166741+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:38.166741+0000","last_peered":"2024-09-16T04:52:38.166741+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:38.166741+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":9595,"log_dups_size":0,"ondisk_log_size":9595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39268352,"num_objects":9587,"num_object_clones":0,"num_object_copies":19174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6250,"num_write_kb":25000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6122},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'9445","reported_seq":9687,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:37.270356+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:37.270356+0000","last_peered":"2024-09-16T04:52:37.270356+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:37.270356+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":9445,"log_dups_size":0,"ondisk_log_size":9445,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38560978,"num_objects":9416,"num_object_clones":0,"num_object_copies":18832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9417,"num_write_kb":37659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":6120}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9440","reported_seq":16296,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:37.221601+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:37.221601+0000","last_peered":"2024-09-16T04:52:37.221601+0000","last_clean":"2024-09-16T04:52:37.221601+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:37.221601+0000","last_undegraded":"2024-09-16T04:52:37.221601+0000","last_fullsized":"2024-09-16T04:52:37.221601+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9440,"log_dups_size":0,"ondisk_log_size":9440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38641664,"num_objects":9434,"num_object_clones":0,"num_object_copies":18868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6163,"num_write_kb":24652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9395","reported_seq":9493,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:37.284648+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:37.284648+0000","last_peered":"2024-09-16T04:52:37.284648+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:37.284648+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9395,"log_dups_size":0,"ondisk_log_size":9395,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38474345,"num_objects":9394,"num_object_clones":0,"num_object_copies":18788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9394,"num_write_kb":37573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5987}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9726","reported_seq":9829,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:37.310222+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:37.310222+0000","last_peered":"2024-09-16T04:52:37.310222+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:37.310222+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9726,"log_dups_size":0,"ondisk_log_size":9726,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39806779,"num_objects":9721,"num_object_clones":0,"num_object_copies":19442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9723,"num_write_kb":38876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":6207}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9578","reported_seq":12028,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.216945+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:38.216945+0000","last_peered":"2024-09-16T04:52:38.216945+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:38.216945+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9578,"log_dups_size":0,"ondisk_log_size":9578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215104,"num_objects":9574,"num_object_clones":0,"num_object_copies":19148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2164,"num_objects_unfound":0,"num_objects_dirty":9574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6332,"num_write_kb":25328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1159,"num_bytes_recovered":4747264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7408},{"shards":"0,2","objects":2164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9696","reported_seq":9836,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.186076+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:38.186076+0000","last_peered":"2024-09-16T04:52:38.186076+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:38.186076+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9696,"log_dups_size":0,"ondisk_log_size":9696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39666281,"num_objects":9685,"num_object_clones":0,"num_object_copies":19370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6465,"num_write_kb":25860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6161},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":623112415,"num_objects":152133,"num_object_clones":0,"num_object_copies":304266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2164,"num_objects_unfound":0,"num_objects_dirty":152133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117367,"num_write_kb":469440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4514,"num_bytes_recovered":18456576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1262628864,"data_stored":1262548847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152289,"ondisk_log_size":152289,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":745628,"kb_used_data":461260,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93626212,"statfs":{"total":96636764160,"available":95873241088,"internally_reserved":0,"allocated":472330240,"data_stored":472061257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":291172009},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593765,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548232,"kb_used_data":309688,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93823608,"statfs":{"total":96636764160,"available":96075374592,"internally_reserved":0,"allocated":317120512,"data_stored":316781949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,0,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746252,"kb_used_data":461948,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93625588,"statfs":{"total":96636764160,"available":95872602112,"internally_reserved":0,"allocated":473034752,"data_stored":472765769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473382912,"data_stored":473379433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316825600,"data_stored":316752541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":472420352,"data_stored":472416873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:40.050 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:40.136 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 223 20 153212 153192 2.68235 1.71094 0.0090717 0.0291176 2024-09-16T04:52:41.059 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:41.061 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:41.117 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-16T04:52:40.362803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":623112434,"num_objects":152135,"num_object_clones":0,"num_object_copies":304270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2164,"num_objects_unfound":0,"num_objects_dirty":152135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117369,"num_write_kb":469442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4515,"num_bytes_recovered":18456595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152292,"ondisk_log_size":152292,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2040112,"kb_used_data":1232896,"kb_used_omap":12,"kb_used_meta":807155,"kb_avail":281075408,"statfs":{"total":289910292480,"available":287821217792,"internally_reserved":0,"allocated":1262485504,"data_stored":1261608975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":826526721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,4,1,13,15,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18788352,"num_objects":4587,"num_object_clones":0,"num_object_copies":9174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-187,"num_objects_unfound":0,"num_objects_dirty":4587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4587,"num_write_kb":18348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":765952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004562"},"pg_stats":[{"pgid":"2.f","version":"47'9361","reported_seq":9440,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.184538+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:38.184538+0000","last_peered":"2024-09-16T04:52:38.184538+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:38.184538+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9361,"log_dups_size":0,"ondisk_log_size":9361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38342656,"num_objects":9361,"num_object_clones":0,"num_object_copies":18722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9361,"num_write_kb":37444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6020},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9446","reported_seq":9562,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:37.331426+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:37.331426+0000","last_peered":"2024-09-16T04:52:37.331426+0000","last_clean":"2024-09-16T04:52:37.331426+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:37.331426+0000","last_undegraded":"2024-09-16T04:52:37.331426+0000","last_fullsized":"2024-09-16T04:52:37.331426+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9446,"log_dups_size":0,"ondisk_log_size":9446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38666240,"num_objects":9440,"num_object_clones":0,"num_object_copies":18880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6415,"num_write_kb":25660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9398","reported_seq":9536,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:37.325270+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:37.325270+0000","last_peered":"2024-09-16T04:52:37.325270+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:37.325270+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9398,"log_dups_size":0,"ondisk_log_size":9398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38477824,"num_objects":9394,"num_object_clones":0,"num_object_copies":18788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6137,"num_write_kb":24548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":6075}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9591","reported_seq":9661,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:37.321170+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:37.321170+0000","last_peered":"2024-09-16T04:52:37.321170+0000","last_clean":"2024-09-16T04:52:37.321170+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:37.321170+0000","last_undegraded":"2024-09-16T04:52:37.321170+0000","last_fullsized":"2024-09-16T04:52:37.321170+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9591,"log_dups_size":0,"ondisk_log_size":9591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9591,"num_write_kb":38364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9551","reported_seq":9770,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:38.121744+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:38.121744+0000","last_peered":"2024-09-16T04:52:38.121744+0000","last_clean":"2024-09-16T04:52:38.121744+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:38.121744+0000","last_undegraded":"2024-09-16T04:52:38.121744+0000","last_fullsized":"2024-09-16T04:52:38.121744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9551,"log_dups_size":0,"ondisk_log_size":9551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38998016,"num_objects":9521,"num_object_clones":0,"num_object_copies":19042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6583,"num_write_kb":26332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9609","reported_seq":9742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.152929+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:38.152929+0000","last_peered":"2024-09-16T04:52:38.152929+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:38.152929+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9609,"log_dups_size":0,"ondisk_log_size":9609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6602,"num_write_kb":26408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6148},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9450","reported_seq":9597,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.172977+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:38.172977+0000","last_peered":"2024-09-16T04:52:38.172977+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:38.172977+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9450,"log_dups_size":0,"ondisk_log_size":9450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38682624,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6105,"num_write_kb":24420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6045},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9503","reported_seq":9718,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:38.173296+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:38.173296+0000","last_peered":"2024-09-16T04:52:38.173296+0000","last_clean":"2024-09-16T04:52:38.173296+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:38.173296+0000","last_undegraded":"2024-09-16T04:52:38.173296+0000","last_fullsized":"2024-09-16T04:52:38.173296+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9503,"log_dups_size":0,"ondisk_log_size":9503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38891520,"num_objects":9495,"num_object_clones":0,"num_object_copies":18990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6573,"num_write_kb":26292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9505","reported_seq":9733,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:37.243593+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:37.243593+0000","last_peered":"2024-09-16T04:52:37.243593+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:37.243593+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9505,"log_dups_size":0,"ondisk_log_size":9505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38797312,"num_objects":9472,"num_object_clones":0,"num_object_copies":18944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6256,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":6151}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'9595","reported_seq":9725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.166741+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:38.166741+0000","last_peered":"2024-09-16T04:52:38.166741+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:38.166741+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":9595,"log_dups_size":0,"ondisk_log_size":9595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39268352,"num_objects":9587,"num_object_clones":0,"num_object_copies":19174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6250,"num_write_kb":25000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6122},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'9445","reported_seq":9687,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:37.270356+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:37.270356+0000","last_peered":"2024-09-16T04:52:37.270356+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:37.270356+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":9445,"log_dups_size":0,"ondisk_log_size":9445,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38560978,"num_objects":9416,"num_object_clones":0,"num_object_copies":18832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9417,"num_write_kb":37659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":6120}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9440","reported_seq":16296,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:37.221601+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:37.221601+0000","last_peered":"2024-09-16T04:52:37.221601+0000","last_clean":"2024-09-16T04:52:37.221601+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:37.221601+0000","last_undegraded":"2024-09-16T04:52:37.221601+0000","last_fullsized":"2024-09-16T04:52:37.221601+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9440,"log_dups_size":0,"ondisk_log_size":9440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38641664,"num_objects":9434,"num_object_clones":0,"num_object_copies":18868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6163,"num_write_kb":24652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9395","reported_seq":9493,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:37.284648+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:37.284648+0000","last_peered":"2024-09-16T04:52:37.284648+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:37.284648+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9395,"log_dups_size":0,"ondisk_log_size":9395,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38474345,"num_objects":9394,"num_object_clones":0,"num_object_copies":18788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9394,"num_write_kb":37573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":5987}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9726","reported_seq":9829,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:37.310222+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:37.310222+0000","last_peered":"2024-09-16T04:52:37.310222+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:37.310222+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9726,"log_dups_size":0,"ondisk_log_size":9726,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39806779,"num_objects":9721,"num_object_clones":0,"num_object_copies":19442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9723,"num_write_kb":38876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":6207}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9578","reported_seq":12028,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.216945+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:38.216945+0000","last_peered":"2024-09-16T04:52:38.216945+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:38.216945+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9578,"log_dups_size":0,"ondisk_log_size":9578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215104,"num_objects":9574,"num_object_clones":0,"num_object_copies":19148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2164,"num_objects_unfound":0,"num_objects_dirty":9574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6332,"num_write_kb":25328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1159,"num_bytes_recovered":4747264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7408},{"shards":"0,2","objects":2164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9696","reported_seq":9836,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.186076+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:38.186076+0000","last_peered":"2024-09-16T04:52:38.186076+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:38.186076+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9696,"log_dups_size":0,"ondisk_log_size":9696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39666281,"num_objects":9685,"num_object_clones":0,"num_object_copies":19370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6465,"num_write_kb":25860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6161},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":623112415,"num_objects":152133,"num_object_clones":0,"num_object_copies":304266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2164,"num_objects_unfound":0,"num_objects_dirty":152133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117367,"num_write_kb":469440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4514,"num_bytes_recovered":18456576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1262628864,"data_stored":1262548847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152289,"ondisk_log_size":152289,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":745628,"kb_used_data":461260,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93626212,"statfs":{"total":96636764160,"available":95873241088,"internally_reserved":0,"allocated":472330240,"data_stored":472061257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":291172009},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593765,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548232,"kb_used_data":309688,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93823608,"statfs":{"total":96636764160,"available":96075374592,"internally_reserved":0,"allocated":317120512,"data_stored":316781949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,0,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746252,"kb_used_data":461948,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93625588,"statfs":{"total":96636764160,"available":95872602112,"internally_reserved":0,"allocated":473034752,"data_stored":472765769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473382912,"data_stored":473379433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316825600,"data_stored":316752541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":472420352,"data_stored":472416873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:41.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:52:41.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'9361", 'reported_seq': 9440, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:38.184538+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:52:38.184538+0000', 'last_peered': '2024-09-16T04:52:38.184538+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:52:38.184538+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 9361, 'log_dups_size': 0, 'ondisk_log_size': 9361, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38342656, 'num_objects': 9361, 'num_object_clones': 0, 'num_object_copies': 18722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9361, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9361, 'num_write_kb': 37444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6020}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:41.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:52:41.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'9398", 'reported_seq': 9536, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:37.325270+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:52:37.325270+0000', 'last_peered': '2024-09-16T04:52:37.325270+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:52:37.325270+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 9398, 'log_dups_size': 0, 'ondisk_log_size': 9398, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38477824, 'num_objects': 9394, 'num_object_clones': 0, 'num_object_copies': 18788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9394, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6137, 'num_write_kb': 24548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 6075}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:41.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:52:41.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'9609", 'reported_seq': 9742, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:38.152929+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:52:38.152929+0000', 'last_peered': '2024-09-16T04:52:38.152929+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:52:38.152929+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 9609, 'log_dups_size': 0, 'ondisk_log_size': 9609, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39337984, 'num_objects': 9604, 'num_object_clones': 0, 'num_object_copies': 19208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9604, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6602, 'num_write_kb': 26408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6148}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:41.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:52:41.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'9450", 'reported_seq': 9597, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:38.172977+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:52:38.172977+0000', 'last_peered': '2024-09-16T04:52:38.172977+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:52:38.172977+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 9450, 'log_dups_size': 0, 'ondisk_log_size': 9450, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38682624, 'num_objects': 9444, 'num_object_clones': 0, 'num_object_copies': 18888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9444, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6105, 'num_write_kb': 24420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6045}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:41.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:52:41.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'9505", 'reported_seq': 9733, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:37.243593+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:52:37.243593+0000', 'last_peered': '2024-09-16T04:52:37.243593+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:52:37.243593+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 9505, 'log_dups_size': 0, 'ondisk_log_size': 9505, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38797312, 'num_objects': 9472, 'num_object_clones': 0, 'num_object_copies': 18944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6256, 'num_write_kb': 25024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 6151}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:41.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:52:41.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'9595", 'reported_seq': 9725, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:38.166741+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:52:38.166741+0000', 'last_peered': '2024-09-16T04:52:38.166741+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:52:38.166741+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 9595, 'log_dups_size': 0, 'ondisk_log_size': 9595, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39268352, 'num_objects': 9587, 'num_object_clones': 0, 'num_object_copies': 19174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9587, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6250, 'num_write_kb': 25000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6122}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:41.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:52:41.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'9445", 'reported_seq': 9687, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:37.270356+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:52:37.270356+0000', 'last_peered': '2024-09-16T04:52:37.270356+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:52:37.270356+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 9445, 'log_dups_size': 0, 'ondisk_log_size': 9445, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38560978, 'num_objects': 9416, 'num_object_clones': 0, 'num_object_copies': 18832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9416, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9417, 'num_write_kb': 37659, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 6120}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:41.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:52:41.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'9395", 'reported_seq': 9493, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:37.284648+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:52:37.284648+0000', 'last_peered': '2024-09-16T04:52:37.284648+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:52:37.284648+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 9395, 'log_dups_size': 0, 'ondisk_log_size': 9395, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38474345, 'num_objects': 9394, 'num_object_clones': 0, 'num_object_copies': 18788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9394, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9394, 'num_write_kb': 37573, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 5987}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:41.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:52:41.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'9726", 'reported_seq': 9829, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:37.310222+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:52:37.310222+0000', 'last_peered': '2024-09-16T04:52:37.310222+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:52:37.310222+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 9726, 'log_dups_size': 0, 'ondisk_log_size': 9726, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39806779, 'num_objects': 9721, 'num_object_clones': 0, 'num_object_copies': 19442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9723, 'num_write_kb': 38876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 6207}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:41.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:52:41.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'9578", 'reported_seq': 12028, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:38.216945+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:52:38.216945+0000', 'last_peered': '2024-09-16T04:52:38.216945+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:52:38.216945+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 9578, 'log_dups_size': 0, 'ondisk_log_size': 9578, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39215104, 'num_objects': 9574, 'num_object_clones': 0, 'num_object_copies': 19148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2164, 'num_objects_unfound': 0, 'num_objects_dirty': 9574, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6332, 'num_write_kb': 25328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1159, 'num_bytes_recovered': 4747264, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7408}, {'shards': '0,2', 'objects': 2164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:41.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:52:41.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'9696", 'reported_seq': 9836, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:38.186076+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:52:38.186076+0000', 'last_peered': '2024-09-16T04:52:38.186076+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:52:38.186076+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 9696, 'log_dups_size': 0, 'ondisk_log_size': 9696, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39666281, 'num_objects': 9685, 'num_object_clones': 0, 'num_object_copies': 19370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9685, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6465, 'num_write_kb': 25860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6161}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:41.121 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:52:41.136 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 224 20 153641 153621 2.67786 1.67578 0.0322668 0.0291654 2024-09-16T04:52:42.143 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 225 20 154071 154051 2.67337 1.67969 0.0856984 0.0292153 2024-09-16T04:52:42.149 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:42.149 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/304270 objects degraded (11.195%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":309,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152135,"data_bytes":623112434,"bytes_used":2089074688,"bytes_avail":287821217792,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":304270,"degraded_ratio":0.11194662635159562,"misplaced_objects":2164,"misplaced_total":304270,"misplaced_ratio":0.0071121043809774217,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":63805,"recovering_keys_per_sec":0,"num_objects_recovered":187,"num_bytes_recovered":765952,"num_keys_recovered":0,"write_bytes_sec":1565100,"read_op_per_sec":0,"write_op_per_sec":382},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:52:42.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:52:42.243 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:43.141 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 226 20 154511 154491 2.66914 1.71875 0.0166321 0.0292612 2024-09-16T04:52:43.208 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:43.208 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:43.271 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-16T04:52:42.363244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":627687666,"num_objects":153252,"num_object_clones":0,"num_object_copies":306504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2164,"num_objects_unfound":0,"num_objects_dirty":153252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":118486,"num_write_kb":473910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4515,"num_bytes_recovered":18456595,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153409,"ondisk_log_size":153409,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2044464,"kb_used_data":1245568,"kb_used_omap":12,"kb_used_meta":798835,"kb_avail":281071056,"statfs":{"total":289910292480,"available":287816761344,"internally_reserved":0,"allocated":1275461632,"data_stored":1274585103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":818007041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,9,7,13,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23363584,"num_objects":5704,"num_object_clones":0,"num_object_copies":11408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-187,"num_objects_unfound":0,"num_objects_dirty":5704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5704,"num_write_kb":22816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":765952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004502"},"pg_stats":[{"pgid":"2.f","version":"47'9361","reported_seq":9440,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.184538+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:38.184538+0000","last_peered":"2024-09-16T04:52:38.184538+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:38.184538+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9361,"log_dups_size":0,"ondisk_log_size":9361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38342656,"num_objects":9361,"num_object_clones":0,"num_object_copies":18722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9361,"num_write_kb":37444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6020},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9581","reported_seq":9697,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:42.262548+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:42.262548+0000","last_peered":"2024-09-16T04:52:42.262548+0000","last_clean":"2024-09-16T04:52:42.262548+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:42.262548+0000","last_undegraded":"2024-09-16T04:52:42.262548+0000","last_fullsized":"2024-09-16T04:52:42.262548+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9581,"log_dups_size":0,"ondisk_log_size":9581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39219200,"num_objects":9575,"num_object_clones":0,"num_object_copies":19150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6550,"num_write_kb":26200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9539","reported_seq":9677,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.117209+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:42.117209+0000","last_peered":"2024-09-16T04:52:42.117209+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:42.117209+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9539,"log_dups_size":0,"ondisk_log_size":9539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39055360,"num_objects":9535,"num_object_clones":0,"num_object_copies":19070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6278,"num_write_kb":25112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":6216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9746","reported_seq":9816,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:42.308840+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:42.308840+0000","last_peered":"2024-09-16T04:52:42.308840+0000","last_clean":"2024-09-16T04:52:42.308840+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:42.308840+0000","last_undegraded":"2024-09-16T04:52:42.308840+0000","last_fullsized":"2024-09-16T04:52:42.308840+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9746,"log_dups_size":0,"ondisk_log_size":9746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39919616,"num_objects":9746,"num_object_clones":0,"num_object_copies":19492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9746,"num_write_kb":38984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9551","reported_seq":9770,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:38.121744+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:38.121744+0000","last_peered":"2024-09-16T04:52:38.121744+0000","last_clean":"2024-09-16T04:52:38.121744+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:38.121744+0000","last_undegraded":"2024-09-16T04:52:38.121744+0000","last_fullsized":"2024-09-16T04:52:38.121744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9551,"log_dups_size":0,"ondisk_log_size":9551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38998016,"num_objects":9521,"num_object_clones":0,"num_object_copies":19042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6583,"num_write_kb":26332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9609","reported_seq":9742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.152929+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:38.152929+0000","last_peered":"2024-09-16T04:52:38.152929+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:38.152929+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9609,"log_dups_size":0,"ondisk_log_size":9609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6602,"num_write_kb":26408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6148},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9450","reported_seq":9597,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.172977+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:38.172977+0000","last_peered":"2024-09-16T04:52:38.172977+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:38.172977+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9450,"log_dups_size":0,"ondisk_log_size":9450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38682624,"num_objects":9444,"num_object_clones":0,"num_object_copies":18888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6105,"num_write_kb":24420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6045},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9503","reported_seq":9718,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:38.173296+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:38.173296+0000","last_peered":"2024-09-16T04:52:38.173296+0000","last_clean":"2024-09-16T04:52:38.173296+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:38.173296+0000","last_undegraded":"2024-09-16T04:52:38.173296+0000","last_fullsized":"2024-09-16T04:52:38.173296+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9503,"log_dups_size":0,"ondisk_log_size":9503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38891520,"num_objects":9495,"num_object_clones":0,"num_object_copies":18990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6573,"num_write_kb":26292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9657","reported_seq":9885,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.331749+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:42.331749+0000","last_peered":"2024-09-16T04:52:42.331749+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:42.331749+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9657,"log_dups_size":0,"ondisk_log_size":9657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39419904,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6408,"num_write_kb":25632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":6303}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'9595","reported_seq":9725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.166741+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:38.166741+0000","last_peered":"2024-09-16T04:52:38.166741+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:38.166741+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":9595,"log_dups_size":0,"ondisk_log_size":9595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39268352,"num_objects":9587,"num_object_clones":0,"num_object_copies":19174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6250,"num_write_kb":25000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6122},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'9598","reported_seq":9840,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.253265+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:42.253265+0000","last_peered":"2024-09-16T04:52:42.253265+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:42.253265+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":9598,"log_dups_size":0,"ondisk_log_size":9598,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39187666,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9570,"num_write_kb":38271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":6273}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9565","reported_seq":16421,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:42.345162+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:42.345162+0000","last_peered":"2024-09-16T04:52:42.345162+0000","last_clean":"2024-09-16T04:52:42.345162+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:42.345162+0000","last_undegraded":"2024-09-16T04:52:42.345162+0000","last_fullsized":"2024-09-16T04:52:42.345162+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9565,"log_dups_size":0,"ondisk_log_size":9565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39153664,"num_objects":9559,"num_object_clones":0,"num_object_copies":19118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6288,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9526","reported_seq":9624,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.262989+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:42.262989+0000","last_peered":"2024-09-16T04:52:42.262989+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:42.262989+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9526,"log_dups_size":0,"ondisk_log_size":9526,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39010921,"num_objects":9525,"num_object_clones":0,"num_object_copies":19050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9525,"num_write_kb":38097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":6118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9851","reported_seq":9954,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.324987+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:42.324987+0000","last_peered":"2024-09-16T04:52:42.324987+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:42.324987+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9851,"log_dups_size":0,"ondisk_log_size":9851,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40318779,"num_objects":9846,"num_object_clones":0,"num_object_copies":19692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9848,"num_write_kb":39376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":6332}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9578","reported_seq":12028,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.216945+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:38.216945+0000","last_peered":"2024-09-16T04:52:38.216945+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:38.216945+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9578,"log_dups_size":0,"ondisk_log_size":9578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215104,"num_objects":9574,"num_object_clones":0,"num_object_copies":19148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2164,"num_objects_unfound":0,"num_objects_dirty":9574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6332,"num_write_kb":25328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1159,"num_bytes_recovered":4747264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7408},{"shards":"0,2","objects":2164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9696","reported_seq":9836,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:38.186076+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:38.186076+0000","last_peered":"2024-09-16T04:52:38.186076+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:38.186076+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9696,"log_dups_size":0,"ondisk_log_size":9696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39666281,"num_objects":9685,"num_object_clones":0,"num_object_copies":19370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6465,"num_write_kb":25860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6161},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":627687647,"num_objects":153250,"num_object_clones":0,"num_object_copies":306500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2164,"num_objects_unfound":0,"num_objects_dirty":153250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":118484,"num_write_kb":473908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4514,"num_bytes_recovered":18456576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1275682816,"data_stored":1275602799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153406,"ondisk_log_size":153406,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":755384,"kb_used_data":466856,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93616456,"statfs":{"total":96636764160,"available":95863250944,"internally_reserved":0,"allocated":478060544,"data_stored":477791561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":295431849},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,4,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593766,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542828,"kb_used_data":316764,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93829012,"statfs":{"total":96636764160,"available":96080908288,"internally_reserved":0,"allocated":324366336,"data_stored":324027773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231468718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746252,"kb_used_data":461948,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93625588,"statfs":{"total":96636764160,"available":95872602112,"internally_reserved":0,"allocated":473034752,"data_stored":472765769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473382912,"data_stored":473379433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324169728,"data_stored":324096669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":478130176,"data_stored":478126697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:44.141 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 227 20 154961 154941 2.66513 1.75781 0.057226 0.029306 2024-09-16T04:52:45.145 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 228 20 155423 155403 2.66135 1.80469 0.0262644 0.0293482 2024-09-16T04:52:46.141 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 229 20 155880 155860 2.65753 1.78516 0.0216674 0.0293901 2024-09-16T04:52:46.273 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:47.142 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 230 20 156318 156298 2.65341 1.71094 0.025365 0.0294369 2024-09-16T04:52:47.317 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:47.319 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:47.407 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-16T04:52:46.364121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632209650,"num_objects":154356,"num_object_clones":0,"num_object_copies":308712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2068,"num_objects_unfound":0,"num_objects_dirty":154356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119590,"num_write_kb":478326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4611,"num_bytes_recovered":18849811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154513,"ondisk_log_size":154513,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2072252,"kb_used_data":1251148,"kb_used_omap":12,"kb_used_meta":821043,"kb_avail":281043268,"statfs":{"total":289910292480,"available":287788306432,"internally_reserved":0,"allocated":1281175552,"data_stored":1280299023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":840748033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,9,13,13,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18481152,"num_objects":4512,"num_object_clones":0,"num_object_copies":9024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-175,"num_objects_unfound":0,"num_objects_dirty":4512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4512,"num_write_kb":18048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":716800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004375"},"pg_stats":[{"pgid":"2.f","version":"47'9503","reported_seq":9582,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:43.252305+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:43.252305+0000","last_peered":"2024-09-16T04:52:43.252305+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:43.252305+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9503,"log_dups_size":0,"ondisk_log_size":9503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38924288,"num_objects":9503,"num_object_clones":0,"num_object_copies":19006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9503,"num_write_kb":38012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6162},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9581","reported_seq":9697,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:42.262548+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:42.262548+0000","last_peered":"2024-09-16T04:52:42.262548+0000","last_clean":"2024-09-16T04:52:42.262548+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:42.262548+0000","last_undegraded":"2024-09-16T04:52:42.262548+0000","last_fullsized":"2024-09-16T04:52:42.262548+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9581,"log_dups_size":0,"ondisk_log_size":9581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39219200,"num_objects":9575,"num_object_clones":0,"num_object_copies":19150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6550,"num_write_kb":26200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9539","reported_seq":9677,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.117209+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:42.117209+0000","last_peered":"2024-09-16T04:52:42.117209+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:42.117209+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9539,"log_dups_size":0,"ondisk_log_size":9539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39055360,"num_objects":9535,"num_object_clones":0,"num_object_copies":19070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6278,"num_write_kb":25112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":6216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9746","reported_seq":9816,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:42.308840+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:42.308840+0000","last_peered":"2024-09-16T04:52:42.308840+0000","last_clean":"2024-09-16T04:52:42.308840+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:42.308840+0000","last_undegraded":"2024-09-16T04:52:42.308840+0000","last_fullsized":"2024-09-16T04:52:42.308840+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9746,"log_dups_size":0,"ondisk_log_size":9746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39919616,"num_objects":9746,"num_object_clones":0,"num_object_copies":19492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9746,"num_write_kb":38984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9701","reported_seq":9920,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:43.175177+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:43.175177+0000","last_peered":"2024-09-16T04:52:43.175177+0000","last_clean":"2024-09-16T04:52:43.175177+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:43.175177+0000","last_undegraded":"2024-09-16T04:52:43.175177+0000","last_fullsized":"2024-09-16T04:52:43.175177+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9701,"log_dups_size":0,"ondisk_log_size":9701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39612416,"num_objects":9671,"num_object_clones":0,"num_object_copies":19342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6733,"num_write_kb":26932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9762","reported_seq":9895,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:43.218823+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:43.218823+0000","last_peered":"2024-09-16T04:52:43.218823+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:43.218823+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9762,"log_dups_size":0,"ondisk_log_size":9762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39964672,"num_objects":9757,"num_object_clones":0,"num_object_copies":19514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6755,"num_write_kb":27020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6301},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9577","reported_seq":9725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:43.154296+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:43.154296+0000","last_peered":"2024-09-16T04:52:43.154296+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:43.154296+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9577,"log_dups_size":0,"ondisk_log_size":9577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39202816,"num_objects":9571,"num_object_clones":0,"num_object_copies":19142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6232,"num_write_kb":24928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6172},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9636","reported_seq":9852,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:43.182194+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:43.182194+0000","last_peered":"2024-09-16T04:52:43.182194+0000","last_clean":"2024-09-16T04:52:43.182194+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:43.182194+0000","last_undegraded":"2024-09-16T04:52:43.182194+0000","last_fullsized":"2024-09-16T04:52:43.182194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9636,"log_dups_size":0,"ondisk_log_size":9636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39436288,"num_objects":9628,"num_object_clones":0,"num_object_copies":19256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6706,"num_write_kb":26824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9657","reported_seq":9885,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.331749+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:42.331749+0000","last_peered":"2024-09-16T04:52:42.331749+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:42.331749+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9657,"log_dups_size":0,"ondisk_log_size":9657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39419904,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6408,"num_write_kb":25632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":6303}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'9725","reported_seq":9854,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:43.189391+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:43.189391+0000","last_peered":"2024-09-16T04:52:43.189391+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:43.189391+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":9725,"log_dups_size":0,"ondisk_log_size":9725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39800832,"num_objects":9717,"num_object_clones":0,"num_object_copies":19434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6380,"num_write_kb":25520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6252},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'9598","reported_seq":9840,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.253265+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:42.253265+0000","last_peered":"2024-09-16T04:52:42.253265+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:42.253265+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":9598,"log_dups_size":0,"ondisk_log_size":9598,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39187666,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9570,"num_write_kb":38271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":6273}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9565","reported_seq":16421,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:42.345162+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:42.345162+0000","last_peered":"2024-09-16T04:52:42.345162+0000","last_clean":"2024-09-16T04:52:42.345162+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:42.345162+0000","last_undegraded":"2024-09-16T04:52:42.345162+0000","last_fullsized":"2024-09-16T04:52:42.345162+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9565,"log_dups_size":0,"ondisk_log_size":9565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39153664,"num_objects":9559,"num_object_clones":0,"num_object_copies":19118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6288,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9526","reported_seq":9624,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.262989+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:42.262989+0000","last_peered":"2024-09-16T04:52:42.262989+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:42.262989+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9526,"log_dups_size":0,"ondisk_log_size":9526,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39010921,"num_objects":9525,"num_object_clones":0,"num_object_copies":19050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9525,"num_write_kb":38097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":6118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9851","reported_seq":9954,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.324987+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:42.324987+0000","last_peered":"2024-09-16T04:52:42.324987+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:42.324987+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9851,"log_dups_size":0,"ondisk_log_size":9851,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40318779,"num_objects":9846,"num_object_clones":0,"num_object_copies":19692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9848,"num_write_kb":39376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":6332}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9710","reported_seq":12352,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:43.265527+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:43.265527+0000","last_peered":"2024-09-16T04:52:43.265527+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:43.265527+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9710,"log_dups_size":0,"ondisk_log_size":9710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39755776,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2068,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6464,"num_write_kb":25856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1255,"num_bytes_recovered":5140480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7636},{"shards":"0,2","objects":2068}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9833","reported_seq":9973,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:43.209279+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:43.209279+0000","last_peered":"2024-09-16T04:52:43.209279+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:43.209279+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9833,"log_dups_size":0,"ondisk_log_size":9833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40227433,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6602,"num_write_kb":26408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6298},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":632209631,"num_objects":154354,"num_object_clones":0,"num_object_copies":308708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2068,"num_objects_unfound":0,"num_objects_dirty":154354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119588,"num_write_kb":478324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4610,"num_bytes_recovered":18849792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1281339392,"data_stored":1281259375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154510,"ondisk_log_size":154510,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":755384,"kb_used_data":466856,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93616456,"statfs":{"total":96636764160,"available":95863250944,"internally_reserved":0,"allocated":478060544,"data_stored":477791561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":295431849},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,4,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593766,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542828,"kb_used_data":316764,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93829012,"statfs":{"total":96636764160,"available":96080908288,"internally_reserved":0,"allocated":324366336,"data_stored":324027773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231468718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774040,"kb_used_data":467528,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93597800,"statfs":{"total":96636764160,"available":95844147200,"internally_reserved":0,"allocated":478748672,"data_stored":478479689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,6,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":479039488,"data_stored":479036009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324169728,"data_stored":324096669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":478130176,"data_stored":478126697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:47.408 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:48.145 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 231 20 156750 156730 2.64919 1.6875 0.0663213 0.0294821 2024-09-16T04:52:48.336 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:48.402 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-16T04:52:46.364121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632209650,"num_objects":154356,"num_object_clones":0,"num_object_copies":308712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2068,"num_objects_unfound":0,"num_objects_dirty":154356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119590,"num_write_kb":478326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4611,"num_bytes_recovered":18849811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154513,"ondisk_log_size":154513,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2072252,"kb_used_data":1251148,"kb_used_omap":12,"kb_used_meta":821043,"kb_avail":281043268,"statfs":{"total":289910292480,"available":287788306432,"internally_reserved":0,"allocated":1281175552,"data_stored":1280299023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":840748033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,9,13,13,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18481152,"num_objects":4512,"num_object_clones":0,"num_object_copies":9024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-175,"num_objects_unfound":0,"num_objects_dirty":4512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4512,"num_write_kb":18048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":716800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004375"},"pg_stats":[{"pgid":"2.f","version":"47'9503","reported_seq":9582,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:43.252305+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:43.252305+0000","last_peered":"2024-09-16T04:52:43.252305+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:43.252305+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9503,"log_dups_size":0,"ondisk_log_size":9503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38924288,"num_objects":9503,"num_object_clones":0,"num_object_copies":19006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9503,"num_write_kb":38012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6162},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9581","reported_seq":9697,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:42.262548+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:42.262548+0000","last_peered":"2024-09-16T04:52:42.262548+0000","last_clean":"2024-09-16T04:52:42.262548+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:42.262548+0000","last_undegraded":"2024-09-16T04:52:42.262548+0000","last_fullsized":"2024-09-16T04:52:42.262548+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9581,"log_dups_size":0,"ondisk_log_size":9581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39219200,"num_objects":9575,"num_object_clones":0,"num_object_copies":19150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6550,"num_write_kb":26200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9539","reported_seq":9677,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.117209+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:42.117209+0000","last_peered":"2024-09-16T04:52:42.117209+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:42.117209+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9539,"log_dups_size":0,"ondisk_log_size":9539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39055360,"num_objects":9535,"num_object_clones":0,"num_object_copies":19070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6278,"num_write_kb":25112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":6216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9746","reported_seq":9816,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:42.308840+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:42.308840+0000","last_peered":"2024-09-16T04:52:42.308840+0000","last_clean":"2024-09-16T04:52:42.308840+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:42.308840+0000","last_undegraded":"2024-09-16T04:52:42.308840+0000","last_fullsized":"2024-09-16T04:52:42.308840+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9746,"log_dups_size":0,"ondisk_log_size":9746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39919616,"num_objects":9746,"num_object_clones":0,"num_object_copies":19492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9746,"num_write_kb":38984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9701","reported_seq":9920,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:43.175177+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:43.175177+0000","last_peered":"2024-09-16T04:52:43.175177+0000","last_clean":"2024-09-16T04:52:43.175177+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:43.175177+0000","last_undegraded":"2024-09-16T04:52:43.175177+0000","last_fullsized":"2024-09-16T04:52:43.175177+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9701,"log_dups_size":0,"ondisk_log_size":9701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39612416,"num_objects":9671,"num_object_clones":0,"num_object_copies":19342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6733,"num_write_kb":26932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9762","reported_seq":9895,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:43.218823+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:43.218823+0000","last_peered":"2024-09-16T04:52:43.218823+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:43.218823+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9762,"log_dups_size":0,"ondisk_log_size":9762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39964672,"num_objects":9757,"num_object_clones":0,"num_object_copies":19514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6755,"num_write_kb":27020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6301},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9577","reported_seq":9725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:43.154296+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:43.154296+0000","last_peered":"2024-09-16T04:52:43.154296+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:43.154296+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9577,"log_dups_size":0,"ondisk_log_size":9577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39202816,"num_objects":9571,"num_object_clones":0,"num_object_copies":19142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6232,"num_write_kb":24928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6172},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9636","reported_seq":9852,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:43.182194+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:43.182194+0000","last_peered":"2024-09-16T04:52:43.182194+0000","last_clean":"2024-09-16T04:52:43.182194+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:43.182194+0000","last_undegraded":"2024-09-16T04:52:43.182194+0000","last_fullsized":"2024-09-16T04:52:43.182194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9636,"log_dups_size":0,"ondisk_log_size":9636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39436288,"num_objects":9628,"num_object_clones":0,"num_object_copies":19256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6706,"num_write_kb":26824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9657","reported_seq":9885,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.331749+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:42.331749+0000","last_peered":"2024-09-16T04:52:42.331749+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:42.331749+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9657,"log_dups_size":0,"ondisk_log_size":9657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39419904,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6408,"num_write_kb":25632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":6303}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'9725","reported_seq":9854,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:43.189391+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:43.189391+0000","last_peered":"2024-09-16T04:52:43.189391+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:43.189391+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":9725,"log_dups_size":0,"ondisk_log_size":9725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39800832,"num_objects":9717,"num_object_clones":0,"num_object_copies":19434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6380,"num_write_kb":25520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6252},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'9598","reported_seq":9840,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.253265+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:42.253265+0000","last_peered":"2024-09-16T04:52:42.253265+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:42.253265+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":9598,"log_dups_size":0,"ondisk_log_size":9598,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39187666,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9570,"num_write_kb":38271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":6273}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9565","reported_seq":16421,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:42.345162+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:42.345162+0000","last_peered":"2024-09-16T04:52:42.345162+0000","last_clean":"2024-09-16T04:52:42.345162+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:42.345162+0000","last_undegraded":"2024-09-16T04:52:42.345162+0000","last_fullsized":"2024-09-16T04:52:42.345162+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9565,"log_dups_size":0,"ondisk_log_size":9565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39153664,"num_objects":9559,"num_object_clones":0,"num_object_copies":19118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6288,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9526","reported_seq":9624,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.262989+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:42.262989+0000","last_peered":"2024-09-16T04:52:42.262989+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:42.262989+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9526,"log_dups_size":0,"ondisk_log_size":9526,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39010921,"num_objects":9525,"num_object_clones":0,"num_object_copies":19050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9525,"num_write_kb":38097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":6118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'9851","reported_seq":9954,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:42.324987+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:42.324987+0000","last_peered":"2024-09-16T04:52:42.324987+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:42.324987+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":9851,"log_dups_size":0,"ondisk_log_size":9851,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40318779,"num_objects":9846,"num_object_clones":0,"num_object_copies":19692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9848,"num_write_kb":39376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":6332}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9710","reported_seq":12352,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:43.265527+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:43.265527+0000","last_peered":"2024-09-16T04:52:43.265527+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:43.265527+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9710,"log_dups_size":0,"ondisk_log_size":9710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39755776,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":2068,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6464,"num_write_kb":25856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1255,"num_bytes_recovered":5140480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7636},{"shards":"0,2","objects":2068}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9833","reported_seq":9973,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:43.209279+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:43.209279+0000","last_peered":"2024-09-16T04:52:43.209279+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:43.209279+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9833,"log_dups_size":0,"ondisk_log_size":9833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40227433,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6602,"num_write_kb":26408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6298},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":632209631,"num_objects":154354,"num_object_clones":0,"num_object_copies":308708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":2068,"num_objects_unfound":0,"num_objects_dirty":154354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119588,"num_write_kb":478324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4610,"num_bytes_recovered":18849792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1281339392,"data_stored":1281259375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154510,"ondisk_log_size":154510,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":755384,"kb_used_data":466856,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93616456,"statfs":{"total":96636764160,"available":95863250944,"internally_reserved":0,"allocated":478060544,"data_stored":477791561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":295431849},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,4,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593766,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542828,"kb_used_data":316764,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93829012,"statfs":{"total":96636764160,"available":96080908288,"internally_reserved":0,"allocated":324366336,"data_stored":324027773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231468718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774040,"kb_used_data":467528,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93597800,"statfs":{"total":96636764160,"available":95844147200,"internally_reserved":0,"allocated":478748672,"data_stored":478479689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,6,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":479039488,"data_stored":479036009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324169728,"data_stored":324096669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":478130176,"data_stored":478126697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:48.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:52:48.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'9503", 'reported_seq': 9582, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:43.252305+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:52:43.252305+0000', 'last_peered': '2024-09-16T04:52:43.252305+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:52:43.252305+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 9503, 'log_dups_size': 0, 'ondisk_log_size': 9503, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38924288, 'num_objects': 9503, 'num_object_clones': 0, 'num_object_copies': 19006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9503, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9503, 'num_write_kb': 38012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6162}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:48.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:52:48.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'9539", 'reported_seq': 9677, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:42.117209+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:52:42.117209+0000', 'last_peered': '2024-09-16T04:52:42.117209+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:52:42.117209+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 9539, 'log_dups_size': 0, 'ondisk_log_size': 9539, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39055360, 'num_objects': 9535, 'num_object_clones': 0, 'num_object_copies': 19070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9535, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6278, 'num_write_kb': 25112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 6216}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:48.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:52:48.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'9762", 'reported_seq': 9895, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:43.218823+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:52:43.218823+0000', 'last_peered': '2024-09-16T04:52:43.218823+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:52:43.218823+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 9762, 'log_dups_size': 0, 'ondisk_log_size': 9762, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39964672, 'num_objects': 9757, 'num_object_clones': 0, 'num_object_copies': 19514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9757, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6755, 'num_write_kb': 27020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6301}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:48.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:52:48.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'9577", 'reported_seq': 9725, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:43.154296+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:52:43.154296+0000', 'last_peered': '2024-09-16T04:52:43.154296+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:52:43.154296+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 9577, 'log_dups_size': 0, 'ondisk_log_size': 9577, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39202816, 'num_objects': 9571, 'num_object_clones': 0, 'num_object_copies': 19142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9571, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6232, 'num_write_kb': 24928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6172}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:48.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:52:48.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'9657", 'reported_seq': 9885, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:42.331749+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:52:42.331749+0000', 'last_peered': '2024-09-16T04:52:42.331749+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:52:42.331749+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 9657, 'log_dups_size': 0, 'ondisk_log_size': 9657, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39419904, 'num_objects': 9624, 'num_object_clones': 0, 'num_object_copies': 19248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6408, 'num_write_kb': 25632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 6303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:48.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:52:48.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'9725", 'reported_seq': 9854, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:43.189391+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:52:43.189391+0000', 'last_peered': '2024-09-16T04:52:43.189391+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:52:43.189391+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 9725, 'log_dups_size': 0, 'ondisk_log_size': 9725, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39800832, 'num_objects': 9717, 'num_object_clones': 0, 'num_object_copies': 19434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9717, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6380, 'num_write_kb': 25520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6252}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:48.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:52:48.404 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'9598", 'reported_seq': 9840, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:42.253265+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:52:42.253265+0000', 'last_peered': '2024-09-16T04:52:42.253265+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:52:42.253265+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 9598, 'log_dups_size': 0, 'ondisk_log_size': 9598, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39187666, 'num_objects': 9569, 'num_object_clones': 0, 'num_object_copies': 19138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9569, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9570, 'num_write_kb': 38271, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 6273}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:48.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:52:48.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'9526", 'reported_seq': 9624, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:42.262989+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:52:42.262989+0000', 'last_peered': '2024-09-16T04:52:42.262989+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:52:42.262989+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 9526, 'log_dups_size': 0, 'ondisk_log_size': 9526, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39010921, 'num_objects': 9525, 'num_object_clones': 0, 'num_object_copies': 19050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9525, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9525, 'num_write_kb': 38097, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 6118}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:48.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:52:48.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'9851", 'reported_seq': 9954, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:42.324987+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:52:42.324987+0000', 'last_peered': '2024-09-16T04:52:42.324987+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:52:42.324987+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 9851, 'log_dups_size': 0, 'ondisk_log_size': 9851, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40318779, 'num_objects': 9846, 'num_object_clones': 0, 'num_object_copies': 19692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9846, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9848, 'num_write_kb': 39376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 6332}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:48.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:52:48.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'9710", 'reported_seq': 12352, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:43.265527+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:52:43.265527+0000', 'last_peered': '2024-09-16T04:52:43.265527+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:52:43.265527+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 9710, 'log_dups_size': 0, 'ondisk_log_size': 9710, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39755776, 'num_objects': 9706, 'num_object_clones': 0, 'num_object_copies': 19412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 2068, 'num_objects_unfound': 0, 'num_objects_dirty': 9706, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6464, 'num_write_kb': 25856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1255, 'num_bytes_recovered': 5140480, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7636}, {'shards': '0,2', 'objects': 2068}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:48.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:52:48.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'9833", 'reported_seq': 9973, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:43.209279+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:52:43.209279+0000', 'last_peered': '2024-09-16T04:52:43.209279+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:52:43.209279+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 9833, 'log_dups_size': 0, 'ondisk_log_size': 9833, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40227433, 'num_objects': 9822, 'num_object_clones': 0, 'num_object_copies': 19644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6602, 'num_write_kb': 26408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6298}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:48.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:52:49.145 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 232 20 157188 157168 2.64515 1.71094 0.0479378 0.0295265 2024-09-16T04:52:49.431 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:49.431 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/313284 objects degraded (10.873%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":317,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156642,"data_bytes":641573106,"bytes_used":2176552960,"bytes_avail":287733739520,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":313284,"degraded_ratio":0.10872562914160953,"misplaced_objects":1953,"misplaced_total":313284,"misplaced_ratio":0.0062339602405485119,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":98950,"recovering_keys_per_sec":0,"num_objects_recovered":290,"num_bytes_recovered":1187840,"num_keys_recovered":0,"write_bytes_sec":2319547,"read_op_per_sec":0,"write_op_per_sec":566},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:52:49.512 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:52:49.513 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:50.145 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 233 20 157611 157591 2.64089 1.65234 0.0253331 0.0295753 2024-09-16T04:52:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:50.412 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:50.451 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-16T04:52:50.365107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641573106,"num_objects":156642,"num_object_clones":0,"num_object_copies":313284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1953,"num_objects_unfound":0,"num_objects_dirty":156642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121876,"num_write_kb":487470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4726,"num_bytes_recovered":19320851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156799,"ondisk_log_size":156799,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2125540,"kb_used_data":1269748,"kb_used_omap":12,"kb_used_meta":855731,"kb_avail":280989980,"statfs":{"total":289910292480,"available":287733739520,"internally_reserved":0,"allocated":1300221952,"data_stored":1299345423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":876268545},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,0,7,7,11,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18460672,"num_objects":4507,"num_object_clones":0,"num_object_copies":9014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-211,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4507,"num_write_kb":18028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":864256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004273"},"pg_stats":[{"pgid":"2.f","version":"47'9650","reported_seq":9729,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:48.307123+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:48.307123+0000","last_peered":"2024-09-16T04:52:48.307123+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:48.307123+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9650,"log_dups_size":0,"ondisk_log_size":9650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39526400,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9650,"num_write_kb":38600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6309},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9730","reported_seq":9846,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:47.393678+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:47.393678+0000","last_peered":"2024-09-16T04:52:47.393678+0000","last_clean":"2024-09-16T04:52:47.393678+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:47.393678+0000","last_undegraded":"2024-09-16T04:52:47.393678+0000","last_fullsized":"2024-09-16T04:52:47.393678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9730,"log_dups_size":0,"ondisk_log_size":9730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39829504,"num_objects":9724,"num_object_clones":0,"num_object_copies":19448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6699,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9677","reported_seq":9815,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:47.316524+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:47.316524+0000","last_peered":"2024-09-16T04:52:47.316524+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:47.316524+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39620608,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6416,"num_write_kb":25664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":6354}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9899","reported_seq":9969,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:47.397756+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:47.397756+0000","last_peered":"2024-09-16T04:52:47.397756+0000","last_clean":"2024-09-16T04:52:47.397756+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:47.397756+0000","last_undegraded":"2024-09-16T04:52:47.397756+0000","last_fullsized":"2024-09-16T04:52:47.397756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9899,"log_dups_size":0,"ondisk_log_size":9899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40546304,"num_objects":9899,"num_object_clones":0,"num_object_copies":19798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9899,"num_write_kb":39596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9849","reported_seq":10068,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:48.242923+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:48.242923+0000","last_peered":"2024-09-16T04:52:48.242923+0000","last_clean":"2024-09-16T04:52:48.242923+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:48.242923+0000","last_undegraded":"2024-09-16T04:52:48.242923+0000","last_fullsized":"2024-09-16T04:52:48.242923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9849,"log_dups_size":0,"ondisk_log_size":9849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40218624,"num_objects":9819,"num_object_clones":0,"num_object_copies":19638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6881,"num_write_kb":27524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9915","reported_seq":10048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:48.187685+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:48.187685+0000","last_peered":"2024-09-16T04:52:48.187685+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:48.187685+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9915,"log_dups_size":0,"ondisk_log_size":9915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40591360,"num_objects":9910,"num_object_clones":0,"num_object_copies":19820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6908,"num_write_kb":27632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6454},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9706","reported_seq":9854,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:48.124980+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:48.124980+0000","last_peered":"2024-09-16T04:52:48.124980+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:48.124980+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9706,"log_dups_size":0,"ondisk_log_size":9706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39731200,"num_objects":9700,"num_object_clones":0,"num_object_copies":19400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6361,"num_write_kb":25444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6301},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9768","reported_seq":9984,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:48.286731+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:48.286731+0000","last_peered":"2024-09-16T04:52:48.286731+0000","last_clean":"2024-09-16T04:52:48.286731+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:48.286731+0000","last_undegraded":"2024-09-16T04:52:48.286731+0000","last_fullsized":"2024-09-16T04:52:48.286731+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9768,"log_dups_size":0,"ondisk_log_size":9768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39976960,"num_objects":9760,"num_object_clones":0,"num_object_copies":19520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6838,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9788","reported_seq":10016,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:47.394822+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:47.394822+0000","last_peered":"2024-09-16T04:52:47.394822+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:47.394822+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9788,"log_dups_size":0,"ondisk_log_size":9788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6539,"num_write_kb":26156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":6434}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'9866","reported_seq":9996,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:48.269500+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:48.269500+0000","last_peered":"2024-09-16T04:52:48.269500+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:48.269500+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":9866,"log_dups_size":0,"ondisk_log_size":9866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40378368,"num_objects":9858,"num_object_clones":0,"num_object_copies":19716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6521,"num_write_kb":26084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6393},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'9755","reported_seq":9997,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:47.354486+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:47.354486+0000","last_peered":"2024-09-16T04:52:47.354486+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:47.354486+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":9755,"log_dups_size":0,"ondisk_log_size":9755,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39830738,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9727,"num_write_kb":38899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":6430}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9699","reported_seq":16556,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:47.360571+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:47.360571+0000","last_peered":"2024-09-16T04:52:47.360571+0000","last_clean":"2024-09-16T04:52:47.360571+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:47.360571+0000","last_undegraded":"2024-09-16T04:52:47.360571+0000","last_fullsized":"2024-09-16T04:52:47.360571+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9699,"log_dups_size":0,"ondisk_log_size":9699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39702528,"num_objects":9693,"num_object_clones":0,"num_object_copies":19386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6422,"num_write_kb":25688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9665","reported_seq":9762,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:47.390405+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:47.390405+0000","last_peered":"2024-09-16T04:52:47.390405+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:47.390405+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9665,"log_dups_size":0,"ondisk_log_size":9665,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39580265,"num_objects":9664,"num_object_clones":0,"num_object_copies":19328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9664,"num_write_kb":38653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":6257}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10015","reported_seq":10117,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:47.339633+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:47.339633+0000","last_peered":"2024-09-16T04:52:47.339633+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:47.339633+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40990523,"num_objects":10010,"num_object_clones":0,"num_object_copies":20020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10012,"num_write_kb":40032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":6496}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9841","reported_seq":12712,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:48.311499+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:48.311499+0000","last_peered":"2024-09-16T04:52:48.311499+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:48.311499+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9841,"log_dups_size":0,"ondisk_log_size":9841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40292352,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1953,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6595,"num_write_kb":26380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1370,"num_bytes_recovered":5611520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7882},{"shards":"0,2","objects":1953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9973","reported_seq":10113,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:48.241294+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:48.241294+0000","last_peered":"2024-09-16T04:52:48.241294+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:48.241294+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9973,"log_dups_size":0,"ondisk_log_size":9973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40800873,"num_objects":9962,"num_object_clones":0,"num_object_copies":19924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6742,"num_write_kb":26968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6438},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641573087,"num_objects":156640,"num_object_clones":0,"num_object_copies":313280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1953,"num_objects_unfound":0,"num_objects_dirty":156640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121874,"num_write_kb":487468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4725,"num_bytes_recovered":19320832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1300328448,"data_stored":1300248431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156796,"ondisk_log_size":156796,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":787488,"kb_used_data":472592,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93584352,"statfs":{"total":96636764160,"available":95830376448,"internally_reserved":0,"allocated":483934208,"data_stored":483665225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":322432681},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,5,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593767,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554224,"kb_used_data":324000,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93817616,"statfs":{"total":96636764160,"available":96069238784,"internally_reserved":0,"allocated":331776000,"data_stored":331437437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783828,"kb_used_data":473156,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93588012,"statfs":{"total":96636764160,"available":95834124288,"internally_reserved":0,"allocated":484511744,"data_stored":484242761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,0,4,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484835328,"data_stored":484831849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331456512,"data_stored":331383453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":484036608,"data_stored":484033129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:51.145 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 234 20 158062 158042 2.63713 1.76172 0.0907464 0.0296169 2024-09-16T04:52:52.148 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 235 20 158541 158521 2.63384 1.87109 0.0732729 0.0296547 2024-09-16T04:52:53.149 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 236 20 158985 158965 2.63003 1.73438 0.0717909 0.0296976 2024-09-16T04:52:53.454 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:54.149 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 237 20 159437 159417 2.62638 1.76562 0.0167985 0.0297385 2024-09-16T04:52:54.358 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:54.358 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:54.442 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-16T04:52:52.365757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641573106,"num_objects":156642,"num_object_clones":0,"num_object_copies":313284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1953,"num_objects_unfound":0,"num_objects_dirty":156642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121876,"num_write_kb":487470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4726,"num_bytes_recovered":19320851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156799,"ondisk_log_size":156799,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2140844,"kb_used_data":1276732,"kb_used_omap":12,"kb_used_meta":864051,"kb_avail":280974676,"statfs":{"total":289910292480,"available":287718068224,"internally_reserved":0,"allocated":1307373568,"data_stored":1306497039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":884788225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,1,5,8,11,14,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18460672,"num_objects":4507,"num_object_clones":0,"num_object_copies":9014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-211,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4507,"num_write_kb":18028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":864256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002953"},"pg_stats":[{"pgid":"2.f","version":"47'9650","reported_seq":9729,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:48.307123+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:48.307123+0000","last_peered":"2024-09-16T04:52:48.307123+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:48.307123+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9650,"log_dups_size":0,"ondisk_log_size":9650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39526400,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9650,"num_write_kb":38600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6309},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9730","reported_seq":9846,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:47.393678+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:47.393678+0000","last_peered":"2024-09-16T04:52:47.393678+0000","last_clean":"2024-09-16T04:52:47.393678+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:47.393678+0000","last_undegraded":"2024-09-16T04:52:47.393678+0000","last_fullsized":"2024-09-16T04:52:47.393678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9730,"log_dups_size":0,"ondisk_log_size":9730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39829504,"num_objects":9724,"num_object_clones":0,"num_object_copies":19448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6699,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9677","reported_seq":9815,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:47.316524+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:47.316524+0000","last_peered":"2024-09-16T04:52:47.316524+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:47.316524+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39620608,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6416,"num_write_kb":25664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":6354}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'9899","reported_seq":9969,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:47.397756+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:47.397756+0000","last_peered":"2024-09-16T04:52:47.397756+0000","last_clean":"2024-09-16T04:52:47.397756+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:47.397756+0000","last_undegraded":"2024-09-16T04:52:47.397756+0000","last_fullsized":"2024-09-16T04:52:47.397756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9899,"log_dups_size":0,"ondisk_log_size":9899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40546304,"num_objects":9899,"num_object_clones":0,"num_object_copies":19798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9899,"num_write_kb":39596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9849","reported_seq":10068,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:48.242923+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:48.242923+0000","last_peered":"2024-09-16T04:52:48.242923+0000","last_clean":"2024-09-16T04:52:48.242923+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:48.242923+0000","last_undegraded":"2024-09-16T04:52:48.242923+0000","last_fullsized":"2024-09-16T04:52:48.242923+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9849,"log_dups_size":0,"ondisk_log_size":9849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40218624,"num_objects":9819,"num_object_clones":0,"num_object_copies":19638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6881,"num_write_kb":27524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'9915","reported_seq":10048,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:48.187685+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:48.187685+0000","last_peered":"2024-09-16T04:52:48.187685+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:48.187685+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":9915,"log_dups_size":0,"ondisk_log_size":9915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40591360,"num_objects":9910,"num_object_clones":0,"num_object_copies":19820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6908,"num_write_kb":27632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6454},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9706","reported_seq":9854,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:48.124980+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:48.124980+0000","last_peered":"2024-09-16T04:52:48.124980+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:48.124980+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9706,"log_dups_size":0,"ondisk_log_size":9706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39731200,"num_objects":9700,"num_object_clones":0,"num_object_copies":19400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6361,"num_write_kb":25444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6301},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9768","reported_seq":9984,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:48.286731+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:48.286731+0000","last_peered":"2024-09-16T04:52:48.286731+0000","last_clean":"2024-09-16T04:52:48.286731+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:48.286731+0000","last_undegraded":"2024-09-16T04:52:48.286731+0000","last_fullsized":"2024-09-16T04:52:48.286731+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9768,"log_dups_size":0,"ondisk_log_size":9768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39976960,"num_objects":9760,"num_object_clones":0,"num_object_copies":19520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6838,"num_write_kb":27352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9788","reported_seq":10016,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:47.394822+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:47.394822+0000","last_peered":"2024-09-16T04:52:47.394822+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:47.394822+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9788,"log_dups_size":0,"ondisk_log_size":9788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6539,"num_write_kb":26156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":6434}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'9866","reported_seq":9996,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:48.269500+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:48.269500+0000","last_peered":"2024-09-16T04:52:48.269500+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:48.269500+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":9866,"log_dups_size":0,"ondisk_log_size":9866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40378368,"num_objects":9858,"num_object_clones":0,"num_object_copies":19716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6521,"num_write_kb":26084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6393},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'9755","reported_seq":9997,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:47.354486+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:47.354486+0000","last_peered":"2024-09-16T04:52:47.354486+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:47.354486+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":9755,"log_dups_size":0,"ondisk_log_size":9755,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39830738,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9727,"num_write_kb":38899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":6430}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9699","reported_seq":16556,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:47.360571+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:47.360571+0000","last_peered":"2024-09-16T04:52:47.360571+0000","last_clean":"2024-09-16T04:52:47.360571+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:47.360571+0000","last_undegraded":"2024-09-16T04:52:47.360571+0000","last_fullsized":"2024-09-16T04:52:47.360571+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9699,"log_dups_size":0,"ondisk_log_size":9699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39702528,"num_objects":9693,"num_object_clones":0,"num_object_copies":19386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6422,"num_write_kb":25688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9665","reported_seq":9762,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:47.390405+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:47.390405+0000","last_peered":"2024-09-16T04:52:47.390405+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:47.390405+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9665,"log_dups_size":0,"ondisk_log_size":9665,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39580265,"num_objects":9664,"num_object_clones":0,"num_object_copies":19328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9664,"num_write_kb":38653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":6257}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10015","reported_seq":10117,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:47.339633+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:47.339633+0000","last_peered":"2024-09-16T04:52:47.339633+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:47.339633+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40990523,"num_objects":10010,"num_object_clones":0,"num_object_copies":20020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10012,"num_write_kb":40032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":6496}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9841","reported_seq":12712,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:48.311499+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:48.311499+0000","last_peered":"2024-09-16T04:52:48.311499+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:48.311499+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9841,"log_dups_size":0,"ondisk_log_size":9841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40292352,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1953,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6595,"num_write_kb":26380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1370,"num_bytes_recovered":5611520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7882},{"shards":"0,2","objects":1953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'9973","reported_seq":10113,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:48.241294+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:48.241294+0000","last_peered":"2024-09-16T04:52:48.241294+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:48.241294+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":9973,"log_dups_size":0,"ondisk_log_size":9973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40800873,"num_objects":9962,"num_object_clones":0,"num_object_copies":19924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6742,"num_write_kb":26968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6438},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641573087,"num_objects":156640,"num_object_clones":0,"num_object_copies":313280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1953,"num_objects_unfound":0,"num_objects_dirty":156640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121874,"num_write_kb":487468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4725,"num_bytes_recovered":19320832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1307504640,"data_stored":1307424623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156796,"ondisk_log_size":156796,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":787488,"kb_used_data":472592,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93584352,"statfs":{"total":96636764160,"available":95830376448,"internally_reserved":0,"allocated":483934208,"data_stored":483665225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":322432681},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,5,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593768,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569528,"kb_used_data":330984,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93802312,"statfs":{"total":96636764160,"available":96053567488,"internally_reserved":0,"allocated":338927616,"data_stored":338589053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,3,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783828,"kb_used_data":473156,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93588012,"statfs":{"total":96636764160,"available":95834124288,"internally_reserved":0,"allocated":484511744,"data_stored":484242761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,0,4,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484835328,"data_stored":484831849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338632704,"data_stored":338559645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":484036608,"data_stored":484033129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:54.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:55.149 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 238 20 159879 159859 2.62259 1.72656 0.0392222 0.0297816 2024-09-16T04:52:55.310 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:55.311 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:55.344 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-16T04:52:54.366233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650797298,"num_objects":158894,"num_object_clones":0,"num_object_copies":317788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1855,"num_objects_unfound":0,"num_objects_dirty":158894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124128,"num_write_kb":496478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4824,"num_bytes_recovered":19722259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158851,"ondisk_log_size":158851,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2127080,"kb_used_data":1287928,"kb_used_omap":12,"kb_used_meta":839091,"kb_avail":280988440,"statfs":{"total":289910292480,"available":287732162560,"internally_reserved":0,"allocated":1318838272,"data_stored":1317961743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":859229185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,10,2,2,6,5,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23109632,"num_objects":5642,"num_object_clones":0,"num_object_copies":11284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-309,"num_objects_unfound":0,"num_objects_dirty":5642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5642,"num_write_kb":22568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":1265664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002988"},"pg_stats":[{"pgid":"2.f","version":"47'9784","reported_seq":9863,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:53.423577+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:53.423577+0000","last_peered":"2024-09-16T04:52:53.423577+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:53.423577+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9784,"log_dups_size":0,"ondisk_log_size":9784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40075264,"num_objects":9784,"num_object_clones":0,"num_object_copies":19568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9784,"num_write_kb":39136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6443},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9874","reported_seq":9990,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:52.417815+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:52.417815+0000","last_peered":"2024-09-16T04:52:52.417815+0000","last_clean":"2024-09-16T04:52:52.417815+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:52.417815+0000","last_undegraded":"2024-09-16T04:52:52.417815+0000","last_fullsized":"2024-09-16T04:52:52.417815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9874,"log_dups_size":0,"ondisk_log_size":9874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40419328,"num_objects":9868,"num_object_clones":0,"num_object_copies":19736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6843,"num_write_kb":27372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9801","reported_seq":9939,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:52.448854+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:52.448854+0000","last_peered":"2024-09-16T04:52:52.448854+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:52.448854+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9801,"log_dups_size":0,"ondisk_log_size":9801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40128512,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6540,"num_write_kb":26160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":6478}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10038","reported_seq":10108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:52.427774+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:52.427774+0000","last_peered":"2024-09-16T04:52:52.427774+0000","last_clean":"2024-09-16T04:52:52.427774+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:52.427774+0000","last_undegraded":"2024-09-16T04:52:52.427774+0000","last_fullsized":"2024-09-16T04:52:52.427774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41115648,"num_objects":10038,"num_object_clones":0,"num_object_copies":20076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10038,"num_write_kb":40152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9998","reported_seq":10216,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:53.357953+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:53.357953+0000","last_peered":"2024-09-16T04:52:53.357953+0000","last_clean":"2024-09-16T04:52:53.357953+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:53.357953+0000","last_undegraded":"2024-09-16T04:52:53.357953+0000","last_fullsized":"2024-09-16T04:52:53.357953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9998,"log_dups_size":0,"ondisk_log_size":9998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40828928,"num_objects":9968,"num_object_clones":0,"num_object_copies":19936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7030,"num_write_kb":28120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10055","reported_seq":10189,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:53.367612+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:53.367612+0000","last_peered":"2024-09-16T04:52:53.367612+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:53.367612+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41164800,"num_objects":10050,"num_object_clones":0,"num_object_copies":20100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7048,"num_write_kb":28192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6594},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9843","reported_seq":9990,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:53.254818+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:53.254818+0000","last_peered":"2024-09-16T04:52:53.254818+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:53.254818+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9843,"log_dups_size":0,"ondisk_log_size":9843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40292352,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6498,"num_write_kb":25992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6438},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9920","reported_seq":10136,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:53.342704+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:53.342704+0000","last_peered":"2024-09-16T04:52:53.342704+0000","last_clean":"2024-09-16T04:52:53.342704+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:53.342704+0000","last_undegraded":"2024-09-16T04:52:53.342704+0000","last_fullsized":"2024-09-16T04:52:53.342704+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9920,"log_dups_size":0,"ondisk_log_size":9920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40599552,"num_objects":9912,"num_object_clones":0,"num_object_copies":19824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6990,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9940","reported_seq":10168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:52.409238+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:52.409238+0000","last_peered":"2024-09-16T04:52:52.409238+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:52.409238+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9940,"log_dups_size":0,"ondisk_log_size":9940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40579072,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6691,"num_write_kb":26764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":6586}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10013","reported_seq":10143,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:53.356880+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:53.356880+0000","last_peered":"2024-09-16T04:52:53.356880+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:53.356880+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40980480,"num_objects":10005,"num_object_clones":0,"num_object_copies":20010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6668,"num_write_kb":26672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6540},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'9877","reported_seq":10119,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:52.387334+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:52.387334+0000","last_peered":"2024-09-16T04:52:52.387334+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:52.387334+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":9877,"log_dups_size":0,"ondisk_log_size":9877,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40330450,"num_objects":9848,"num_object_clones":0,"num_object_copies":19696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9849,"num_write_kb":39387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":6552}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9851","reported_seq":16708,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:52.393010+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:52.393010+0000","last_peered":"2024-09-16T04:52:52.393010+0000","last_clean":"2024-09-16T04:52:52.393010+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:52.393010+0000","last_undegraded":"2024-09-16T04:52:52.393010+0000","last_fullsized":"2024-09-16T04:52:52.393010+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9851,"log_dups_size":0,"ondisk_log_size":9851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6574,"num_write_kb":26296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9817","reported_seq":9911,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:52.480576+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:52.480576+0000","last_peered":"2024-09-16T04:52:52.480576+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:52.480576+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9817,"log_dups_size":0,"ondisk_log_size":9817,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40202857,"num_objects":9816,"num_object_clones":0,"num_object_copies":19632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9816,"num_write_kb":39261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":6409}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10136","reported_seq":10239,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:52.420575+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:52.420575+0000","last_peered":"2024-09-16T04:52:52.420575+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:52.420575+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41486139,"num_objects":10131,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10133,"num_write_kb":40516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":6617}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9999","reported_seq":13067,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:53.397352+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:53.397352+0000","last_peered":"2024-09-16T04:52:53.397352+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:53.397352+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9999,"log_dups_size":0,"ondisk_log_size":9999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40939520,"num_objects":9995,"num_object_clones":0,"num_object_copies":19990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1855,"num_objects_unfound":0,"num_objects_dirty":9995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6753,"num_write_kb":27012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1468,"num_bytes_recovered":6012928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8138},{"shards":"0,2","objects":1855}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'10102","reported_seq":10242,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:53.382181+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:53.382181+0000","last_peered":"2024-09-16T04:52:53.382181+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:53.382181+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41329257,"num_objects":10091,"num_object_clones":0,"num_object_copies":20182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6871,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6567},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650797279,"num_objects":158892,"num_object_clones":0,"num_object_copies":317784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1855,"num_objects_unfound":0,"num_objects_dirty":158892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124126,"num_write_kb":496476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4823,"num_bytes_recovered":19722240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1319211008,"data_stored":1319130991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158848,"ondisk_log_size":158848,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780544,"kb_used_data":478128,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93591296,"statfs":{"total":96636764160,"available":95837487104,"internally_reserved":0,"allocated":489603072,"data_stored":489334089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,3,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593768,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569528,"kb_used_data":330984,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93802312,"statfs":{"total":96636764160,"available":96053567488,"internally_reserved":0,"allocated":338927616,"data_stored":338589053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,3,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":777008,"kb_used_data":478816,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93594832,"statfs":{"total":96636764160,"available":95841107968,"internally_reserved":0,"allocated":490307584,"data_stored":490038601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,1,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":490725376,"data_stored":490721897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338632704,"data_stored":338559645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":489852928,"data_stored":489849449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:52:55.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:52:55.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'9784", 'reported_seq': 9863, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:53.423577+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:52:53.423577+0000', 'last_peered': '2024-09-16T04:52:53.423577+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:52:53.423577+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 9784, 'log_dups_size': 0, 'ondisk_log_size': 9784, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40075264, 'num_objects': 9784, 'num_object_clones': 0, 'num_object_copies': 19568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9784, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9784, 'num_write_kb': 39136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6443}, {'shards': '0,2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:55.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:52:55.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'9801", 'reported_seq': 9939, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:52.448854+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:52:52.448854+0000', 'last_peered': '2024-09-16T04:52:52.448854+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:52:52.448854+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 9801, 'log_dups_size': 0, 'ondisk_log_size': 9801, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40128512, 'num_objects': 9797, 'num_object_clones': 0, 'num_object_copies': 19594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9797, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6540, 'num_write_kb': 26160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3319}, {'shards': '2', 'objects': 6478}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:55.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:52:55.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'10055", 'reported_seq': 10189, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:53.367612+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:52:53.367612+0000', 'last_peered': '2024-09-16T04:52:53.367612+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:52:53.367612+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41164800, 'num_objects': 10050, 'num_object_clones': 0, 'num_object_copies': 20100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10050, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7048, 'num_write_kb': 28192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6594}, {'shards': '0,2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:55.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:52:55.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'9843", 'reported_seq': 9990, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:53.254818+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:52:53.254818+0000', 'last_peered': '2024-09-16T04:52:53.254818+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:52:53.254818+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 9843, 'log_dups_size': 0, 'ondisk_log_size': 9843, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40292352, 'num_objects': 9837, 'num_object_clones': 0, 'num_object_copies': 19674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9837, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6498, 'num_write_kb': 25992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6438}, {'shards': '0,2', 'objects': 3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:55.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:52:55.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'9940", 'reported_seq': 10168, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:52.409238+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:52:52.409238+0000', 'last_peered': '2024-09-16T04:52:52.409238+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:52:52.409238+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 9940, 'log_dups_size': 0, 'ondisk_log_size': 9940, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40579072, 'num_objects': 9907, 'num_object_clones': 0, 'num_object_copies': 19814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9907, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6691, 'num_write_kb': 26764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3321}, {'shards': '2', 'objects': 6586}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:55.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:52:55.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'10013", 'reported_seq': 10143, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:53.356880+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:52:53.356880+0000', 'last_peered': '2024-09-16T04:52:53.356880+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:52:53.356880+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40980480, 'num_objects': 10005, 'num_object_clones': 0, 'num_object_copies': 20010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10005, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6668, 'num_write_kb': 26672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6540}, {'shards': '0,2', 'objects': 3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:55.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:52:55.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'9877", 'reported_seq': 10119, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:52.387334+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:52:52.387334+0000', 'last_peered': '2024-09-16T04:52:52.387334+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:52:52.387334+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 9877, 'log_dups_size': 0, 'ondisk_log_size': 9877, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40330450, 'num_objects': 9848, 'num_object_clones': 0, 'num_object_copies': 19696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9848, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9849, 'num_write_kb': 39387, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3296}, {'shards': '2', 'objects': 6552}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:55.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:52:55.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'9817", 'reported_seq': 9911, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:52.480576+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:52:52.480576+0000', 'last_peered': '2024-09-16T04:52:52.480576+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:52:52.480576+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 9817, 'log_dups_size': 0, 'ondisk_log_size': 9817, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40202857, 'num_objects': 9816, 'num_object_clones': 0, 'num_object_copies': 19632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9816, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9816, 'num_write_kb': 39261, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3407}, {'shards': '2', 'objects': 6409}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:55.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:52:55.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'10136", 'reported_seq': 10239, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:52.420575+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:52:52.420575+0000', 'last_peered': '2024-09-16T04:52:52.420575+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:52:52.420575+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41486139, 'num_objects': 10131, 'num_object_clones': 0, 'num_object_copies': 20262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10133, 'num_write_kb': 40516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3514}, {'shards': '2', 'objects': 6617}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:52:55.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:52:55.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'9999", 'reported_seq': 13067, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:53.397352+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:52:53.397352+0000', 'last_peered': '2024-09-16T04:52:53.397352+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:52:53.397352+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 9999, 'log_dups_size': 0, 'ondisk_log_size': 9999, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40939520, 'num_objects': 9995, 'num_object_clones': 0, 'num_object_copies': 19990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1855, 'num_objects_unfound': 0, 'num_objects_dirty': 9995, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6753, 'num_write_kb': 27012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1468, 'num_bytes_recovered': 6012928, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8138}, {'shards': '0,2', 'objects': 1855}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:55.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:52:55.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'10102", 'reported_seq': 10242, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:53.382181+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:52:53.382181+0000', 'last_peered': '2024-09-16T04:52:53.382181+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:52:53.382181+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41329257, 'num_objects': 10091, 'num_object_clones': 0, 'num_object_copies': 20182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10091, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6871, 'num_write_kb': 27484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6567}, {'shards': '0,2', 'objects': 3524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:52:55.349 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:52:56.149 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout: 239 20 160311 160291 2.61868 1.6875 0.0378507 0.0298277 2024-09-16T04:52:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:56.514 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/317788 objects degraded (10.718%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":324,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158894,"data_bytes":650797298,"bytes_used":2178129920,"bytes_avail":287732162560,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":317788,"degraded_ratio":0.1071846639898297,"misplaced_objects":1855,"misplaced_total":317788,"misplaced_ratio":0.0058372248165443627,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":105445,"recovering_keys_per_sec":0,"num_objects_recovered":309,"num_bytes_recovered":1265664,"num_keys_recovered":0,"write_bytes_sec":1925323,"read_op_per_sec":0,"write_op_per_sec":470},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:52:56.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:52:56.591 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:52:57.150 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Total time run: 240.028 2024-09-16T04:52:57.150 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Total writes made: 160690 2024-09-16T04:52:57.150 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Write size: 4096 2024-09-16T04:52:57.150 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Object size: 4096 2024-09-16T04:52:57.150 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Bandwidth (MB/sec): 2.61509 2024-09-16T04:52:57.150 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Stddev Bandwidth: 1.66897 2024-09-16T04:52:57.150 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Max bandwidth (MB/sec): 10.5938 2024-09-16T04:52:57.151 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Min bandwidth (MB/sec): 0 2024-09-16T04:52:57.151 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Average IOPS: 669 2024-09-16T04:52:57.151 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Stddev IOPS: 427.262 2024-09-16T04:52:57.151 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Max IOPS: 2712 2024-09-16T04:52:57.151 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Min IOPS: 0 2024-09-16T04:52:57.151 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Average Latency(s): 0.0298697 2024-09-16T04:52:57.151 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Stddev Latency(s): 0.0493212 2024-09-16T04:52:57.151 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Max latency(s): 2.69414 2024-09-16T04:52:57.151 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Min latency(s): 0.00206892 2024-09-16T04:52:57.161 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Cleaning up (deleting benchmark objects) 2024-09-16T04:52:57.322 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:52:57.323 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:52:57.376 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-16T04:52:56.366756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650797298,"num_objects":158894,"num_object_clones":0,"num_object_copies":317788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1855,"num_objects_unfound":0,"num_objects_dirty":158894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124128,"num_write_kb":496478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4824,"num_bytes_recovered":19722259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158851,"ondisk_log_size":158851,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2127080,"kb_used_data":1287928,"kb_used_omap":12,"kb_used_meta":839091,"kb_avail":280988440,"statfs":{"total":289910292480,"available":287732162560,"internally_reserved":0,"allocated":1318838272,"data_stored":1317961743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":859229185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,10,2,2,6,5,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18587648,"num_objects":4538,"num_object_clones":0,"num_object_copies":9076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-213,"num_objects_unfound":0,"num_objects_dirty":4538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4538,"num_write_kb":18152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":872448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003006"},"pg_stats":[{"pgid":"2.f","version":"47'9784","reported_seq":9863,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:53.423577+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:53.423577+0000","last_peered":"2024-09-16T04:52:53.423577+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:53.423577+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9784,"log_dups_size":0,"ondisk_log_size":9784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40075264,"num_objects":9784,"num_object_clones":0,"num_object_copies":19568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9784,"num_write_kb":39136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6443},{"shards":"0,2","objects":3340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'9874","reported_seq":9990,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:52.417815+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:52.417815+0000","last_peered":"2024-09-16T04:52:52.417815+0000","last_clean":"2024-09-16T04:52:52.417815+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:52.417815+0000","last_undegraded":"2024-09-16T04:52:52.417815+0000","last_fullsized":"2024-09-16T04:52:52.417815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":9874,"log_dups_size":0,"ondisk_log_size":9874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40419328,"num_objects":9868,"num_object_clones":0,"num_object_copies":19736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6843,"num_write_kb":27372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9801","reported_seq":9939,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:52.448854+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:52.448854+0000","last_peered":"2024-09-16T04:52:52.448854+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:52.448854+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9801,"log_dups_size":0,"ondisk_log_size":9801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40128512,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6540,"num_write_kb":26160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3319},{"shards":"2","objects":6478}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10038","reported_seq":10108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:52.427774+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:52.427774+0000","last_peered":"2024-09-16T04:52:52.427774+0000","last_clean":"2024-09-16T04:52:52.427774+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:52.427774+0000","last_undegraded":"2024-09-16T04:52:52.427774+0000","last_fullsized":"2024-09-16T04:52:52.427774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41115648,"num_objects":10038,"num_object_clones":0,"num_object_copies":20076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10038,"num_write_kb":40152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'9998","reported_seq":10216,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:53.357953+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:53.357953+0000","last_peered":"2024-09-16T04:52:53.357953+0000","last_clean":"2024-09-16T04:52:53.357953+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:53.357953+0000","last_undegraded":"2024-09-16T04:52:53.357953+0000","last_fullsized":"2024-09-16T04:52:53.357953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":9998,"log_dups_size":0,"ondisk_log_size":9998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40828928,"num_objects":9968,"num_object_clones":0,"num_object_copies":19936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7030,"num_write_kb":28120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10055","reported_seq":10189,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:53.367612+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:53.367612+0000","last_peered":"2024-09-16T04:52:53.367612+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:53.367612+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41164800,"num_objects":10050,"num_object_clones":0,"num_object_copies":20100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7048,"num_write_kb":28192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6594},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9843","reported_seq":9990,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:53.254818+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:53.254818+0000","last_peered":"2024-09-16T04:52:53.254818+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:53.254818+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9843,"log_dups_size":0,"ondisk_log_size":9843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40292352,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6498,"num_write_kb":25992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6438},{"shards":"0,2","objects":3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'9920","reported_seq":10136,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:53.342704+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:53.342704+0000","last_peered":"2024-09-16T04:52:53.342704+0000","last_clean":"2024-09-16T04:52:53.342704+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:53.342704+0000","last_undegraded":"2024-09-16T04:52:53.342704+0000","last_fullsized":"2024-09-16T04:52:53.342704+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":9920,"log_dups_size":0,"ondisk_log_size":9920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40599552,"num_objects":9912,"num_object_clones":0,"num_object_copies":19824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6990,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'9940","reported_seq":10168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:52.409238+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:52.409238+0000","last_peered":"2024-09-16T04:52:52.409238+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:52.409238+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":9940,"log_dups_size":0,"ondisk_log_size":9940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40579072,"num_objects":9907,"num_object_clones":0,"num_object_copies":19814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6691,"num_write_kb":26764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3321},{"shards":"2","objects":6586}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149520+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149520+0000","last_peered":"2024-09-16T04:51:33.149520+0000","last_clean":"2024-09-16T04:51:33.149520+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149520+0000","last_undegraded":"2024-09-16T04:51:33.149520+0000","last_fullsized":"2024-09-16T04:51:33.149520+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10013","reported_seq":10143,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:53.356880+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:53.356880+0000","last_peered":"2024-09-16T04:52:53.356880+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:53.356880+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40980480,"num_objects":10005,"num_object_clones":0,"num_object_copies":20010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6668,"num_write_kb":26672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6540},{"shards":"0,2","objects":3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'9877","reported_seq":10119,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:52.387334+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:52.387334+0000","last_peered":"2024-09-16T04:52:52.387334+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:52.387334+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":9877,"log_dups_size":0,"ondisk_log_size":9877,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40330450,"num_objects":9848,"num_object_clones":0,"num_object_copies":19696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9849,"num_write_kb":39387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3296},{"shards":"2","objects":6552}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9851","reported_seq":16708,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:52.393010+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:52.393010+0000","last_peered":"2024-09-16T04:52:52.393010+0000","last_clean":"2024-09-16T04:52:52.393010+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:52.393010+0000","last_undegraded":"2024-09-16T04:52:52.393010+0000","last_fullsized":"2024-09-16T04:52:52.393010+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9851,"log_dups_size":0,"ondisk_log_size":9851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6574,"num_write_kb":26296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9817","reported_seq":9911,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:52.480576+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:52.480576+0000","last_peered":"2024-09-16T04:52:52.480576+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:52.480576+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9817,"log_dups_size":0,"ondisk_log_size":9817,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40202857,"num_objects":9816,"num_object_clones":0,"num_object_copies":19632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9816,"num_write_kb":39261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3407},{"shards":"2","objects":6409}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10136","reported_seq":10239,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:52.420575+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:52.420575+0000","last_peered":"2024-09-16T04:52:52.420575+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:52.420575+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41486139,"num_objects":10131,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10133,"num_write_kb":40516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3514},{"shards":"2","objects":6617}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153827+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153827+0000","last_peered":"2024-09-16T04:51:33.153827+0000","last_clean":"2024-09-16T04:51:33.153827+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153827+0000","last_undegraded":"2024-09-16T04:51:33.153827+0000","last_fullsized":"2024-09-16T04:51:33.153827+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'9999","reported_seq":13067,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:53.397352+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:53.397352+0000","last_peered":"2024-09-16T04:52:53.397352+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:53.397352+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":9999,"log_dups_size":0,"ondisk_log_size":9999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40939520,"num_objects":9995,"num_object_clones":0,"num_object_copies":19990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1855,"num_objects_unfound":0,"num_objects_dirty":9995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6753,"num_write_kb":27012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1468,"num_bytes_recovered":6012928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8138},{"shards":"0,2","objects":1855}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'10102","reported_seq":10242,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:53.382181+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:53.382181+0000","last_peered":"2024-09-16T04:52:53.382181+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:53.382181+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41329257,"num_objects":10091,"num_object_clones":0,"num_object_copies":20182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6871,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6567},{"shards":"0,2","objects":3524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650797279,"num_objects":158892,"num_object_clones":0,"num_object_copies":317784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1855,"num_objects_unfound":0,"num_objects_dirty":158892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124126,"num_write_kb":496476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4823,"num_bytes_recovered":19722240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1319211008,"data_stored":1319130991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158848,"ondisk_log_size":158848,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":780544,"kb_used_data":478128,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93591296,"statfs":{"total":96636764160,"available":95837487104,"internally_reserved":0,"allocated":489603072,"data_stored":489334089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,3,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593768,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569528,"kb_used_data":330984,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93802312,"statfs":{"total":96636764160,"available":96053567488,"internally_reserved":0,"allocated":338927616,"data_stored":338589053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,3,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":777008,"kb_used_data":478816,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93594832,"statfs":{"total":96636764160,"available":95841107968,"internally_reserved":0,"allocated":490307584,"data_stored":490038601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,1,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":490725376,"data_stored":490721897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338632704,"data_stored":338559645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":489852928,"data_stored":489849449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:00.378 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:01.181 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:01.182 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:01.217 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-16T04:53:00.367706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656670986,"num_objects":160329,"num_object_clones":0,"num_object_copies":320658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1750,"num_objects_unfound":0,"num_objects_dirty":160329,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":126305,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4929,"num_bytes_recovered":20152339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160328,"ondisk_log_size":160328,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2139620,"kb_used_data":1304628,"kb_used_omap":12,"kb_used_meta":834931,"kb_avail":280975900,"statfs":{"total":289910292480,"available":287719321600,"internally_reserved":0,"allocated":1335939072,"data_stored":1335062543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":854969345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,8,13,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15097880,"num_objects":3687,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-203,"num_objects_unfound":0,"num_objects_dirty":3687,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4429,"num_write_kb":16229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":831488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003129"},"pg_stats":[{"pgid":"2.f","version":"47'9912","reported_seq":9992,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.245597+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:58.245597+0000","last_peered":"2024-09-16T04:52:58.245597+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:58.245597+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9912,"log_dups_size":0,"ondisk_log_size":9912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40247296,"num_objects":9826,"num_object_clones":0,"num_object_copies":19652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9912,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6485},{"shards":"0,2","objects":3297}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10013","reported_seq":10128,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:57.475152+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:57.475152+0000","last_peered":"2024-09-16T04:52:57.475152+0000","last_clean":"2024-09-16T04:52:57.475152+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:57.475152+0000","last_undegraded":"2024-09-16T04:52:57.475152+0000","last_fullsized":"2024-09-16T04:52:57.475152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40882176,"num_objects":9981,"num_object_clones":0,"num_object_copies":19962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6982,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9940","reported_seq":10079,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.400761+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:57.400761+0000","last_peered":"2024-09-16T04:52:57.400761+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:57.400761+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9940,"log_dups_size":0,"ondisk_log_size":9940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620056,"num_objects":9918,"num_object_clones":0,"num_object_copies":19836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6679,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3310},{"shards":"2","objects":6599}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10186","reported_seq":10256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:57.467350+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:57.467350+0000","last_peered":"2024-09-16T04:52:57.467350+0000","last_clean":"2024-09-16T04:52:57.467350+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:57.467350+0000","last_undegraded":"2024-09-16T04:52:57.467350+0000","last_fullsized":"2024-09-16T04:52:57.467350+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41607168,"num_objects":10158,"num_object_clones":0,"num_object_copies":20316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10186,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10116","reported_seq":10335,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:58.314807+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:58.314807+0000","last_peered":"2024-09-16T04:52:58.314807+0000","last_clean":"2024-09-16T04:52:58.314807+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:58.314807+0000","last_undegraded":"2024-09-16T04:52:58.314807+0000","last_fullsized":"2024-09-16T04:52:58.314807+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41074688,"num_objects":10028,"num_object_clones":0,"num_object_copies":20056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7148,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10184","reported_seq":10318,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.352047+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:58.352047+0000","last_peered":"2024-09-16T04:52:58.352047+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:58.352047+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41488384,"num_objects":10129,"num_object_clones":0,"num_object_copies":20258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7177,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6673},{"shards":"0,2","objects":3431}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9967","reported_seq":10116,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.325118+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:58.325118+0000","last_peered":"2024-09-16T04:52:58.325118+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:58.325118+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9967,"log_dups_size":0,"ondisk_log_size":9967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40521728,"num_objects":9893,"num_object_clones":0,"num_object_copies":19786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6622,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6494},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10045","reported_seq":10260,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:58.402407+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:58.402407+0000","last_peered":"2024-09-16T04:52:58.402407+0000","last_clean":"2024-09-16T04:52:58.402407+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:58.402407+0000","last_undegraded":"2024-09-16T04:52:58.402407+0000","last_fullsized":"2024-09-16T04:52:58.402407+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40841216,"num_objects":9971,"num_object_clones":0,"num_object_copies":19942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7115,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10063","reported_seq":10291,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.443808+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:57.443808+0000","last_peered":"2024-09-16T04:52:57.443808+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:57.443808+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41033728,"num_objects":10018,"num_object_clones":0,"num_object_copies":20036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6814,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3315},{"shards":"2","objects":6697}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10160","reported_seq":10290,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.422934+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:58.422934+0000","last_peered":"2024-09-16T04:52:58.422934+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:58.422934+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41197568,"num_objects":10058,"num_object_clones":0,"num_object_copies":20116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6815,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6593},{"shards":"0,2","objects":3418}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10010","reported_seq":10252,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.431452+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:57.431452+0000","last_peered":"2024-09-16T04:52:57.431452+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:57.431452+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40834258,"num_objects":9971,"num_object_clones":0,"num_object_copies":19942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9982,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3291},{"shards":"2","objects":6675}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9998","reported_seq":16855,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:57.489091+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:57.489091+0000","last_peered":"2024-09-16T04:52:57.489091+0000","last_clean":"2024-09-16T04:52:57.489091+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:57.489091+0000","last_undegraded":"2024-09-16T04:52:57.489091+0000","last_fullsized":"2024-09-16T04:52:57.489091+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9998,"log_dups_size":0,"ondisk_log_size":9998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40878080,"num_objects":9980,"num_object_clones":0,"num_object_copies":19960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6721,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9947","reported_seq":10045,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.421370+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:57.421370+0000","last_peered":"2024-09-16T04:52:57.421370+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:57.421370+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9947,"log_dups_size":0,"ondisk_log_size":9947,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40669801,"num_objects":9930,"num_object_clones":0,"num_object_copies":19860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9946,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3399},{"shards":"2","objects":6523}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10280","reported_seq":10383,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.475486+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:57.475486+0000","last_peered":"2024-09-16T04:52:57.475486+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:57.475486+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41994043,"num_objects":10255,"num_object_clones":0,"num_object_copies":20510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10277,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3504},{"shards":"2","objects":6741}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10147","reported_seq":13426,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.419701+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:58.419701+0000","last_peered":"2024-09-16T04:52:58.419701+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:58.419701+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1750,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6901,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1573,"num_bytes_recovered":6443008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8299},{"shards":"0,2","objects":1750}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'10257","reported_seq":10397,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.371470+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:58.371470+0000","last_peered":"2024-09-16T04:52:58.371470+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:58.371470+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41611881,"num_objects":10160,"num_object_clones":0,"num_object_copies":20320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7026,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6636},{"shards":"0,2","objects":3481}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":656670967,"num_objects":160327,"num_object_clones":0,"num_object_copies":320654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1750,"num_objects_unfound":0,"num_objects_dirty":160327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":126303,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4928,"num_bytes_recovered":20152320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1333751808,"data_stored":1333667719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160325,"ondisk_log_size":160325,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":790172,"kb_used_data":483596,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93581668,"statfs":{"total":96636764160,"available":95827628032,"internally_reserved":0,"allocated":495202304,"data_stored":494933321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593769,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564208,"kb_used_data":338144,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93807632,"statfs":{"total":96636764160,"available":96059015168,"internally_reserved":0,"allocated":346259456,"data_stored":345920893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231468718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785240,"kb_used_data":482888,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93586600,"statfs":{"total":96636764160,"available":95832678400,"internally_reserved":0,"allocated":494477312,"data_stored":494208329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":493129728,"data_stored":493126249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":346013696,"data_stored":345940637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":494608384,"data_stored":494600833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:01.218 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:02.211 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:02.211 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:02.269 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-16T04:53:00.367706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656670986,"num_objects":160329,"num_object_clones":0,"num_object_copies":320658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1750,"num_objects_unfound":0,"num_objects_dirty":160329,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":126305,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4929,"num_bytes_recovered":20152339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160328,"ondisk_log_size":160328,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2139620,"kb_used_data":1304628,"kb_used_omap":12,"kb_used_meta":834931,"kb_avail":280975900,"statfs":{"total":289910292480,"available":287719321600,"internally_reserved":0,"allocated":1335939072,"data_stored":1335062543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":854969345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,8,13,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15097880,"num_objects":3687,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-203,"num_objects_unfound":0,"num_objects_dirty":3687,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4429,"num_write_kb":16229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":831488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003129"},"pg_stats":[{"pgid":"2.f","version":"47'9912","reported_seq":9992,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.245597+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:58.245597+0000","last_peered":"2024-09-16T04:52:58.245597+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:58.245597+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9912,"log_dups_size":0,"ondisk_log_size":9912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40247296,"num_objects":9826,"num_object_clones":0,"num_object_copies":19652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9912,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6485},{"shards":"0,2","objects":3297}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10013","reported_seq":10128,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:57.475152+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:57.475152+0000","last_peered":"2024-09-16T04:52:57.475152+0000","last_clean":"2024-09-16T04:52:57.475152+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:57.475152+0000","last_undegraded":"2024-09-16T04:52:57.475152+0000","last_fullsized":"2024-09-16T04:52:57.475152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40882176,"num_objects":9981,"num_object_clones":0,"num_object_copies":19962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6982,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9940","reported_seq":10079,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.400761+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:57.400761+0000","last_peered":"2024-09-16T04:52:57.400761+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:57.400761+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9940,"log_dups_size":0,"ondisk_log_size":9940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620056,"num_objects":9918,"num_object_clones":0,"num_object_copies":19836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6679,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3310},{"shards":"2","objects":6599}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10186","reported_seq":10256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:57.467350+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:57.467350+0000","last_peered":"2024-09-16T04:52:57.467350+0000","last_clean":"2024-09-16T04:52:57.467350+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:57.467350+0000","last_undegraded":"2024-09-16T04:52:57.467350+0000","last_fullsized":"2024-09-16T04:52:57.467350+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41607168,"num_objects":10158,"num_object_clones":0,"num_object_copies":20316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10186,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10116","reported_seq":10335,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:58.314807+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:58.314807+0000","last_peered":"2024-09-16T04:52:58.314807+0000","last_clean":"2024-09-16T04:52:58.314807+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:58.314807+0000","last_undegraded":"2024-09-16T04:52:58.314807+0000","last_fullsized":"2024-09-16T04:52:58.314807+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41074688,"num_objects":10028,"num_object_clones":0,"num_object_copies":20056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7148,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10184","reported_seq":10318,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.352047+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:58.352047+0000","last_peered":"2024-09-16T04:52:58.352047+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:58.352047+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41488384,"num_objects":10129,"num_object_clones":0,"num_object_copies":20258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7177,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6673},{"shards":"0,2","objects":3431}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9967","reported_seq":10116,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.325118+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:58.325118+0000","last_peered":"2024-09-16T04:52:58.325118+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:58.325118+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9967,"log_dups_size":0,"ondisk_log_size":9967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40521728,"num_objects":9893,"num_object_clones":0,"num_object_copies":19786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6622,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6494},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10045","reported_seq":10260,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:58.402407+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:58.402407+0000","last_peered":"2024-09-16T04:52:58.402407+0000","last_clean":"2024-09-16T04:52:58.402407+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:58.402407+0000","last_undegraded":"2024-09-16T04:52:58.402407+0000","last_fullsized":"2024-09-16T04:52:58.402407+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40841216,"num_objects":9971,"num_object_clones":0,"num_object_copies":19942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7115,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10063","reported_seq":10291,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.443808+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:57.443808+0000","last_peered":"2024-09-16T04:52:57.443808+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:57.443808+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41033728,"num_objects":10018,"num_object_clones":0,"num_object_copies":20036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6814,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3315},{"shards":"2","objects":6697}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10160","reported_seq":10290,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.422934+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:58.422934+0000","last_peered":"2024-09-16T04:52:58.422934+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:58.422934+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41197568,"num_objects":10058,"num_object_clones":0,"num_object_copies":20116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6815,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6593},{"shards":"0,2","objects":3418}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10010","reported_seq":10252,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.431452+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:57.431452+0000","last_peered":"2024-09-16T04:52:57.431452+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:57.431452+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40834258,"num_objects":9971,"num_object_clones":0,"num_object_copies":19942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9982,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3291},{"shards":"2","objects":6675}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9998","reported_seq":16855,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:57.489091+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:57.489091+0000","last_peered":"2024-09-16T04:52:57.489091+0000","last_clean":"2024-09-16T04:52:57.489091+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:57.489091+0000","last_undegraded":"2024-09-16T04:52:57.489091+0000","last_fullsized":"2024-09-16T04:52:57.489091+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9998,"log_dups_size":0,"ondisk_log_size":9998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40878080,"num_objects":9980,"num_object_clones":0,"num_object_copies":19960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6721,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9947","reported_seq":10045,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.421370+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:57.421370+0000","last_peered":"2024-09-16T04:52:57.421370+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:57.421370+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9947,"log_dups_size":0,"ondisk_log_size":9947,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40669801,"num_objects":9930,"num_object_clones":0,"num_object_copies":19860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9946,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3399},{"shards":"2","objects":6523}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10280","reported_seq":10383,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.475486+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:57.475486+0000","last_peered":"2024-09-16T04:52:57.475486+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:57.475486+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41994043,"num_objects":10255,"num_object_clones":0,"num_object_copies":20510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10277,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3504},{"shards":"2","objects":6741}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10147","reported_seq":13426,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.419701+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:58.419701+0000","last_peered":"2024-09-16T04:52:58.419701+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:58.419701+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1750,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6901,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1573,"num_bytes_recovered":6443008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8299},{"shards":"0,2","objects":1750}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'10257","reported_seq":10397,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.371470+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:58.371470+0000","last_peered":"2024-09-16T04:52:58.371470+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:58.371470+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41611881,"num_objects":10160,"num_object_clones":0,"num_object_copies":20320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7026,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6636},{"shards":"0,2","objects":3481}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":656670967,"num_objects":160327,"num_object_clones":0,"num_object_copies":320654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1750,"num_objects_unfound":0,"num_objects_dirty":160327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":126303,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4928,"num_bytes_recovered":20152320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1333751808,"data_stored":1333667719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160325,"ondisk_log_size":160325,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":790172,"kb_used_data":483596,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93581668,"statfs":{"total":96636764160,"available":95827628032,"internally_reserved":0,"allocated":495202304,"data_stored":494933321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593769,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564208,"kb_used_data":338144,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93807632,"statfs":{"total":96636764160,"available":96059015168,"internally_reserved":0,"allocated":346259456,"data_stored":345920893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231468718},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785240,"kb_used_data":482888,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93586600,"statfs":{"total":96636764160,"available":95832678400,"internally_reserved":0,"allocated":494477312,"data_stored":494208329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":493129728,"data_stored":493126249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":346013696,"data_stored":345940637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":494608384,"data_stored":494600833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:02.270 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:53:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'9912", 'reported_seq': 9992, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:58.245597+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:52:58.245597+0000', 'last_peered': '2024-09-16T04:52:58.245597+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:52:58.245597+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 9912, 'log_dups_size': 0, 'ondisk_log_size': 9912, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40247296, 'num_objects': 9826, 'num_object_clones': 0, 'num_object_copies': 19652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9912, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6485}, {'shards': '0,2', 'objects': 3297}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:53:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'9940", 'reported_seq': 10079, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:57.400761+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:52:57.400761+0000', 'last_peered': '2024-09-16T04:52:57.400761+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:52:57.400761+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 9940, 'log_dups_size': 0, 'ondisk_log_size': 9940, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620056, 'num_objects': 9918, 'num_object_clones': 0, 'num_object_copies': 19836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9918, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6679, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3310}, {'shards': '2', 'objects': 6599}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:53:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'10184", 'reported_seq': 10318, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:58.352047+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:52:58.352047+0000', 'last_peered': '2024-09-16T04:52:58.352047+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:52:58.352047+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41488384, 'num_objects': 10129, 'num_object_clones': 0, 'num_object_copies': 20258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10129, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7177, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6673}, {'shards': '0,2', 'objects': 3431}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:53:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'9967", 'reported_seq': 10116, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:58.325118+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:52:58.325118+0000', 'last_peered': '2024-09-16T04:52:58.325118+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:52:58.325118+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 9967, 'log_dups_size': 0, 'ondisk_log_size': 9967, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40521728, 'num_objects': 9893, 'num_object_clones': 0, 'num_object_copies': 19786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6622, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6494}, {'shards': '0,2', 'objects': 3365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:53:02.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'10063", 'reported_seq': 10291, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:57.443808+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:52:57.443808+0000', 'last_peered': '2024-09-16T04:52:57.443808+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:52:57.443808+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41033728, 'num_objects': 10018, 'num_object_clones': 0, 'num_object_copies': 20036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10018, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6814, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3315}, {'shards': '2', 'objects': 6697}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:02.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:53:02.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'10160", 'reported_seq': 10290, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:58.422934+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:52:58.422934+0000', 'last_peered': '2024-09-16T04:52:58.422934+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:52:58.422934+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41197568, 'num_objects': 10058, 'num_object_clones': 0, 'num_object_copies': 20116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10058, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6815, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6593}, {'shards': '0,2', 'objects': 3418}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:02.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:53:02.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'10010", 'reported_seq': 10252, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:57.431452+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:52:57.431452+0000', 'last_peered': '2024-09-16T04:52:57.431452+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:52:57.431452+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40834258, 'num_objects': 9971, 'num_object_clones': 0, 'num_object_copies': 19942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9971, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9982, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3291}, {'shards': '2', 'objects': 6675}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:02.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:53:02.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'9947", 'reported_seq': 10045, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:57.421370+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:52:57.421370+0000', 'last_peered': '2024-09-16T04:52:57.421370+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:52:57.421370+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 9947, 'log_dups_size': 0, 'ondisk_log_size': 9947, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40669801, 'num_objects': 9930, 'num_object_clones': 0, 'num_object_copies': 19860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9930, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9946, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3399}, {'shards': '2', 'objects': 6523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:02.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:53:02.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'10280", 'reported_seq': 10383, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:57.475486+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:52:57.475486+0000', 'last_peered': '2024-09-16T04:52:57.475486+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:52:57.475486+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41994043, 'num_objects': 10255, 'num_object_clones': 0, 'num_object_copies': 20510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10255, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10277, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3504}, {'shards': '2', 'objects': 6741}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:02.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:53:02.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'10147", 'reported_seq': 13426, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:58.419701+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:52:58.419701+0000', 'last_peered': '2024-09-16T04:52:58.419701+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:52:58.419701+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41168896, 'num_objects': 10051, 'num_object_clones': 0, 'num_object_copies': 20102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1750, 'num_objects_unfound': 0, 'num_objects_dirty': 10051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6901, 'num_write_kb': 27420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1573, 'num_bytes_recovered': 6443008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8299}, {'shards': '0,2', 'objects': 1750}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:02.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:53:02.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'10257", 'reported_seq': 10397, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:52:58.371470+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:52:58.371470+0000', 'last_peered': '2024-09-16T04:52:58.371470+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:52:58.371470+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41611881, 'num_objects': 10160, 'num_object_clones': 0, 'num_object_copies': 20320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7026, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6636}, {'shards': '0,2', 'objects': 3481}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:02.273 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:53:03.323 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:03.323 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/320658 objects degraded (10.623%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":331,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":160329,"data_bytes":656670986,"bytes_used":2218348544,"bytes_avail":287691943936,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":320658,"degraded_ratio":0.10622532417716071,"misplaced_objects":1750,"misplaced_total":320658,"misplaced_ratio":0.0054575279581360828,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":69272,"recovering_keys_per_sec":0,"num_objects_recovered":203,"num_bytes_recovered":831488,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":1384515,"read_op_per_sec":0,"write_op_per_sec":368},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:53:03.380 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:53:03.380 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:04.265 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:04.266 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:04.328 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-16T04:53:02.368220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656670986,"num_objects":160329,"num_object_clones":0,"num_object_copies":320658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1750,"num_objects_unfound":0,"num_objects_dirty":160329,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":126305,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4929,"num_bytes_recovered":20152339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160328,"ondisk_log_size":160328,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2166356,"kb_used_data":1304996,"kb_used_omap":12,"kb_used_meta":861299,"kb_avail":280949164,"statfs":{"total":289910292480,"available":287691943936,"internally_reserved":0,"allocated":1336315904,"data_stored":1335435303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":881970177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,8,9,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15097880,"num_objects":3687,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-203,"num_objects_unfound":0,"num_objects_dirty":3687,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4429,"num_write_kb":16229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":831488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003112"},"pg_stats":[{"pgid":"2.f","version":"47'9912","reported_seq":9992,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.245597+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:52:58.245597+0000","last_peered":"2024-09-16T04:52:58.245597+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:52:58.245597+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9912,"log_dups_size":0,"ondisk_log_size":9912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40247296,"num_objects":9826,"num_object_clones":0,"num_object_copies":19652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9912,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6485},{"shards":"0,2","objects":3297}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10013","reported_seq":10128,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:57.475152+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:52:57.475152+0000","last_peered":"2024-09-16T04:52:57.475152+0000","last_clean":"2024-09-16T04:52:57.475152+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:52:57.475152+0000","last_undegraded":"2024-09-16T04:52:57.475152+0000","last_fullsized":"2024-09-16T04:52:57.475152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40882176,"num_objects":9981,"num_object_clones":0,"num_object_copies":19962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6982,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'9940","reported_seq":10079,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.400761+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:52:57.400761+0000","last_peered":"2024-09-16T04:52:57.400761+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:52:57.400761+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":9940,"log_dups_size":0,"ondisk_log_size":9940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620056,"num_objects":9918,"num_object_clones":0,"num_object_copies":19836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9918,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6679,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3310},{"shards":"2","objects":6599}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10186","reported_seq":10256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:57.467350+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:52:57.467350+0000","last_peered":"2024-09-16T04:52:57.467350+0000","last_clean":"2024-09-16T04:52:57.467350+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:52:57.467350+0000","last_undegraded":"2024-09-16T04:52:57.467350+0000","last_fullsized":"2024-09-16T04:52:57.467350+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41607168,"num_objects":10158,"num_object_clones":0,"num_object_copies":20316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10186,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10116","reported_seq":10335,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:58.314807+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:52:58.314807+0000","last_peered":"2024-09-16T04:52:58.314807+0000","last_clean":"2024-09-16T04:52:58.314807+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:52:58.314807+0000","last_undegraded":"2024-09-16T04:52:58.314807+0000","last_fullsized":"2024-09-16T04:52:58.314807+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41074688,"num_objects":10028,"num_object_clones":0,"num_object_copies":20056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7148,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10184","reported_seq":10318,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.352047+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:52:58.352047+0000","last_peered":"2024-09-16T04:52:58.352047+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:52:58.352047+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41488384,"num_objects":10129,"num_object_clones":0,"num_object_copies":20258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7177,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6673},{"shards":"0,2","objects":3431}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'9967","reported_seq":10116,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.325118+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:52:58.325118+0000","last_peered":"2024-09-16T04:52:58.325118+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:52:58.325118+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":9967,"log_dups_size":0,"ondisk_log_size":9967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40521728,"num_objects":9893,"num_object_clones":0,"num_object_copies":19786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6622,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6494},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10045","reported_seq":10260,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:58.402407+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:52:58.402407+0000","last_peered":"2024-09-16T04:52:58.402407+0000","last_clean":"2024-09-16T04:52:58.402407+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:52:58.402407+0000","last_undegraded":"2024-09-16T04:52:58.402407+0000","last_fullsized":"2024-09-16T04:52:58.402407+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40841216,"num_objects":9971,"num_object_clones":0,"num_object_copies":19942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7115,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10063","reported_seq":10291,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.443808+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:52:57.443808+0000","last_peered":"2024-09-16T04:52:57.443808+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:52:57.443808+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41033728,"num_objects":10018,"num_object_clones":0,"num_object_copies":20036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6814,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3315},{"shards":"2","objects":6697}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10160","reported_seq":10290,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.422934+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:52:58.422934+0000","last_peered":"2024-09-16T04:52:58.422934+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:52:58.422934+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41197568,"num_objects":10058,"num_object_clones":0,"num_object_copies":20116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6815,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6593},{"shards":"0,2","objects":3418}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10010","reported_seq":10252,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.431452+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:52:57.431452+0000","last_peered":"2024-09-16T04:52:57.431452+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:52:57.431452+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40834258,"num_objects":9971,"num_object_clones":0,"num_object_copies":19942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9982,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3291},{"shards":"2","objects":6675}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'9998","reported_seq":16855,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:52:57.489091+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:52:57.489091+0000","last_peered":"2024-09-16T04:52:57.489091+0000","last_clean":"2024-09-16T04:52:57.489091+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:52:57.489091+0000","last_undegraded":"2024-09-16T04:52:57.489091+0000","last_fullsized":"2024-09-16T04:52:57.489091+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":9998,"log_dups_size":0,"ondisk_log_size":9998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40878080,"num_objects":9980,"num_object_clones":0,"num_object_copies":19960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6721,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'9947","reported_seq":10045,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.421370+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:52:57.421370+0000","last_peered":"2024-09-16T04:52:57.421370+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:52:57.421370+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":9947,"log_dups_size":0,"ondisk_log_size":9947,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40669801,"num_objects":9930,"num_object_clones":0,"num_object_copies":19860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9946,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3399},{"shards":"2","objects":6523}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10280","reported_seq":10383,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:57.475486+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:52:57.475486+0000","last_peered":"2024-09-16T04:52:57.475486+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:52:57.475486+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41994043,"num_objects":10255,"num_object_clones":0,"num_object_copies":20510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10277,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3504},{"shards":"2","objects":6741}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10147","reported_seq":13426,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.419701+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:52:58.419701+0000","last_peered":"2024-09-16T04:52:58.419701+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:52:58.419701+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1750,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6901,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1573,"num_bytes_recovered":6443008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8299},{"shards":"0,2","objects":1750}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'10257","reported_seq":10397,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:52:58.371470+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:52:58.371470+0000","last_peered":"2024-09-16T04:52:58.371470+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:52:58.371470+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41611881,"num_objects":10160,"num_object_clones":0,"num_object_copies":20320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7026,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6636},{"shards":"0,2","objects":3481}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":656670967,"num_objects":160327,"num_object_clones":0,"num_object_copies":320654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1750,"num_objects_unfound":0,"num_objects_dirty":160327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":126303,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4928,"num_bytes_recovered":20152320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1333972992,"data_stored":1333884831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160325,"ondisk_log_size":160325,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":790172,"kb_used_data":483596,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93581668,"statfs":{"total":96636764160,"available":95827628032,"internally_reserved":0,"allocated":495202304,"data_stored":494933321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593770,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590944,"kb_used_data":338512,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93780896,"statfs":{"total":96636764160,"available":96031637504,"internally_reserved":0,"allocated":346636288,"data_stored":346293653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":258469550},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785240,"kb_used_data":482888,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93586600,"statfs":{"total":96636764160,"available":95832678400,"internally_reserved":0,"allocated":494477312,"data_stored":494208329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":493129728,"data_stored":493126249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":346234880,"data_stored":346157749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":494608384,"data_stored":494600833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:07.330 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:08.202 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:08.203 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:08.274 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-16T04:53:06.369229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647315722,"num_objects":158045,"num_object_clones":0,"num_object_copies":316090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1636,"num_objects_unfound":0,"num_objects_dirty":158045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128589,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5043,"num_bytes_recovered":20619283,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2168120,"kb_used_data":1294280,"kb_used_omap":12,"kb_used_meta":873779,"kb_avail":280947400,"statfs":{"total":289910292480,"available":287690137600,"internally_reserved":0,"allocated":1325342720,"data_stored":1324458047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":894749697},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,9,9,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3481576,"num_objects":-849,"num_object_clones":0,"num_object_copies":-1698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":-849,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4461,"num_write_kb":7221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002996"},"pg_stats":[{"pgid":"2.f","version":"47'10070","reported_seq":10150,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:03.427941+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:03.427941+0000","last_peered":"2024-09-16T04:53:03.427941+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:03.427941+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39600128,"num_objects":9668,"num_object_clones":0,"num_object_copies":19336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10070,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6327},{"shards":"0,2","objects":3139}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10151","reported_seq":10267,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:02.488678+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:02.488678+0000","last_peered":"2024-09-16T04:53:02.488678+0000","last_clean":"2024-09-16T04:53:02.488678+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:02.488678+0000","last_undegraded":"2024-09-16T04:53:02.488678+0000","last_fullsized":"2024-09-16T04:53:02.488678+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40316928,"num_objects":9843,"num_object_clones":0,"num_object_copies":19686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7120,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10085","reported_seq":10224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:02.456441+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:02.456441+0000","last_peered":"2024-09-16T04:53:02.456441+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:02.456441+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40026136,"num_objects":9773,"num_object_clones":0,"num_object_copies":19546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6824,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3165},{"shards":"2","objects":6454}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10314","reported_seq":10384,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:02.448127+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:02.448127+0000","last_peered":"2024-09-16T04:53:02.448127+0000","last_clean":"2024-09-16T04:53:02.448127+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:02.448127+0000","last_undegraded":"2024-09-16T04:53:02.448127+0000","last_fullsized":"2024-09-16T04:53:02.448127+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41082880,"num_objects":10030,"num_object_clones":0,"num_object_copies":20060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10314,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10257","reported_seq":10476,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:03.378556+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:03.378556+0000","last_peered":"2024-09-16T04:53:03.378556+0000","last_clean":"2024-09-16T04:53:03.378556+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:03.378556+0000","last_undegraded":"2024-09-16T04:53:03.378556+0000","last_fullsized":"2024-09-16T04:53:03.378556+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40497152,"num_objects":9887,"num_object_clones":0,"num_object_copies":19774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7289,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10324","reported_seq":10458,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:03.427245+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:03.427245+0000","last_peered":"2024-09-16T04:53:03.427245+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:03.427245+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40914944,"num_objects":9989,"num_object_clones":0,"num_object_copies":19978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7317,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6533},{"shards":"0,2","objects":3291}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10119","reported_seq":10268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:03.472406+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:03.472406+0000","last_peered":"2024-09-16T04:53:03.472406+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:03.472406+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6774,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6342},{"shards":"0,2","objects":3213}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10187","reported_seq":10403,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:03.464844+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:03.464844+0000","last_peered":"2024-09-16T04:53:03.464844+0000","last_clean":"2024-09-16T04:53:03.464844+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:03.464844+0000","last_undegraded":"2024-09-16T04:53:03.464844+0000","last_fullsized":"2024-09-16T04:53:03.464844+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40259584,"num_objects":9829,"num_object_clones":0,"num_object_copies":19658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7257,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10200","reported_seq":10428,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:02.484959+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:02.484959+0000","last_peered":"2024-09-16T04:53:02.484959+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:02.484959+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40472576,"num_objects":9881,"num_object_clones":0,"num_object_copies":19762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6951,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3178},{"shards":"2","objects":6560}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10301","reported_seq":10431,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:03.425294+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:03.425294+0000","last_peered":"2024-09-16T04:53:03.425294+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:03.425294+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620032,"num_objects":9917,"num_object_clones":0,"num_object_copies":19834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6956,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6452},{"shards":"0,2","objects":3277}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10151","reported_seq":10393,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:02.454722+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:02.454722+0000","last_peered":"2024-09-16T04:53:02.454722+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:02.454722+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40256722,"num_objects":9830,"num_object_clones":0,"num_object_copies":19660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10123,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3150},{"shards":"2","objects":6534}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10131","reported_seq":16988,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:02.448303+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:02.448303+0000","last_peered":"2024-09-16T04:53:02.448303+0000","last_clean":"2024-09-16T04:53:02.448303+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:02.448303+0000","last_undegraded":"2024-09-16T04:53:02.448303+0000","last_fullsized":"2024-09-16T04:53:02.448303+0000","mapping_epoch":46,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40333312,"num_objects":9847,"num_object_clones":0,"num_object_copies":19694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6854,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10093","reported_seq":10191,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:02.428618+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:02.428618+0000","last_peered":"2024-09-16T04:53:02.428618+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:02.428618+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40071785,"num_objects":9784,"num_object_clones":0,"num_object_copies":19568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10092,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3253},{"shards":"2","objects":6377}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10417","reported_seq":10519,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:02.528161+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:02.528161+0000","last_peered":"2024-09-16T04:53:02.528161+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:02.528161+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41432891,"num_objects":10118,"num_object_clones":0,"num_object_copies":20236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10414,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3367},{"shards":"2","objects":6604}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10287","reported_seq":13794,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:03.460316+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:03.460316+0000","last_peered":"2024-09-16T04:53:03.460316+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:03.460316+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40595456,"num_objects":9911,"num_object_clones":0,"num_object_copies":19822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1636,"num_objects_unfound":0,"num_objects_dirty":9911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7041,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1687,"num_bytes_recovered":6909952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8273},{"shards":"0,2","objects":1636}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'10422","reported_seq":10562,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:03.391386+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:03.391386+0000","last_peered":"2024-09-16T04:53:03.391386+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:03.391386+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40936041,"num_objects":9995,"num_object_clones":0,"num_object_copies":19990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7191,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6471},{"shards":"0,2","objects":3316}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":647315703,"num_objects":158043,"num_object_clones":0,"num_object_copies":316086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1636,"num_objects_unfound":0,"num_objects_dirty":158043,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128587,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5042,"num_bytes_recovered":20619264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1322373120,"data_stored":1322284959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":789292,"kb_used_data":478556,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93582548,"statfs":{"total":96636764160,"available":95828529152,"internally_reserved":0,"allocated":490041344,"data_stored":489768289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593770,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590944,"kb_used_data":338512,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93780896,"statfs":{"total":96636764160,"available":96031637504,"internally_reserved":0,"allocated":346636288,"data_stored":346293653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":258469550},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":787884,"kb_used_data":477212,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93583956,"statfs":{"total":96636764160,"available":95829970944,"internally_reserved":0,"allocated":488665088,"data_stored":488396105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":487194624,"data_stored":487191145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":346234880,"data_stored":346157749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":488943616,"data_stored":488936065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:08.275 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:09.052 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:09.105 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-16T04:53:08.369672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":642498826,"num_objects":156869,"num_object_clones":0,"num_object_copies":313738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1636,"num_objects_unfound":0,"num_objects_dirty":156869,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129765,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5043,"num_bytes_recovered":20619283,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160888,"ondisk_log_size":160888,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2153400,"kb_used_data":1287876,"kb_used_omap":12,"kb_used_meta":865459,"kb_avail":280962120,"statfs":{"total":289910292480,"available":287705210880,"internally_reserved":0,"allocated":1318785024,"data_stored":1317900351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":886230017},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,6,3,11,5,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8298472,"num_objects":-2025,"num_object_clones":0,"num_object_copies":-4050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":-2025,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5637,"num_write_kb":7221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002916"},"pg_stats":[{"pgid":"2.f","version":"47'10070","reported_seq":10150,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:03.427941+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:03.427941+0000","last_peered":"2024-09-16T04:53:03.427941+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:03.427941+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39600128,"num_objects":9668,"num_object_clones":0,"num_object_copies":19336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10070,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6327},{"shards":"0,2","objects":3139}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10295","reported_seq":10411,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:07.478147+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:07.478147+0000","last_peered":"2024-09-16T04:53:07.478147+0000","last_clean":"2024-09-16T04:53:07.478147+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:07.478147+0000","last_undegraded":"2024-09-16T04:53:07.478147+0000","last_fullsized":"2024-09-16T04:53:07.478147+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39727104,"num_objects":9699,"num_object_clones":0,"num_object_copies":19398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7264,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10225","reported_seq":10363,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:07.545946+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:07.545946+0000","last_peered":"2024-09-16T04:53:07.545946+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:07.545946+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39452696,"num_objects":9633,"num_object_clones":0,"num_object_copies":19266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6964,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3025},{"shards":"2","objects":6314}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10454","reported_seq":10524,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:07.493124+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:07.493124+0000","last_peered":"2024-09-16T04:53:07.493124+0000","last_clean":"2024-09-16T04:53:07.493124+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:07.493124+0000","last_undegraded":"2024-09-16T04:53:07.493124+0000","last_fullsized":"2024-09-16T04:53:07.493124+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40509440,"num_objects":9890,"num_object_clones":0,"num_object_copies":19780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10454,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10257","reported_seq":10476,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:03.378556+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:03.378556+0000","last_peered":"2024-09-16T04:53:03.378556+0000","last_clean":"2024-09-16T04:53:03.378556+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:03.378556+0000","last_undegraded":"2024-09-16T04:53:03.378556+0000","last_fullsized":"2024-09-16T04:53:03.378556+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40497152,"num_objects":9887,"num_object_clones":0,"num_object_copies":19774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7289,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10324","reported_seq":10458,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:03.427245+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:03.427245+0000","last_peered":"2024-09-16T04:53:03.427245+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:03.427245+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40914944,"num_objects":9989,"num_object_clones":0,"num_object_copies":19978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7317,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6533},{"shards":"0,2","objects":3291}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10119","reported_seq":10268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:03.472406+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:03.472406+0000","last_peered":"2024-09-16T04:53:03.472406+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:03.472406+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6774,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6342},{"shards":"0,2","objects":3213}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10187","reported_seq":10403,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:03.464844+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:03.464844+0000","last_peered":"2024-09-16T04:53:03.464844+0000","last_clean":"2024-09-16T04:53:03.464844+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:03.464844+0000","last_undegraded":"2024-09-16T04:53:03.464844+0000","last_fullsized":"2024-09-16T04:53:03.464844+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40259584,"num_objects":9829,"num_object_clones":0,"num_object_copies":19658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7257,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10353","reported_seq":10581,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:07.485320+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:07.485320+0000","last_peered":"2024-09-16T04:53:07.485320+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:07.485320+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39845888,"num_objects":9728,"num_object_clones":0,"num_object_copies":19456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7104,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3025},{"shards":"2","objects":6407}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10301","reported_seq":10431,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:03.425294+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:03.425294+0000","last_peered":"2024-09-16T04:53:03.425294+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:03.425294+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620032,"num_objects":9917,"num_object_clones":0,"num_object_copies":19834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6956,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6452},{"shards":"0,2","objects":3277}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10299","reported_seq":10541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:07.439983+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:07.439983+0000","last_peered":"2024-09-16T04:53:07.439983+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:07.439983+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39650514,"num_objects":9682,"num_object_clones":0,"num_object_copies":19364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10271,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3002},{"shards":"2","objects":6386}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10286","reported_seq":17143,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:07.486667+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:07.486667+0000","last_peered":"2024-09-16T04:53:07.486667+0000","last_clean":"2024-09-16T04:53:07.486667+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:07.486667+0000","last_undegraded":"2024-09-16T04:53:07.486667+0000","last_fullsized":"2024-09-16T04:53:07.486667+0000","mapping_epoch":46,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39698432,"num_objects":9692,"num_object_clones":0,"num_object_copies":19384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7009,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10256","reported_seq":10354,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:07.536856+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:07.536856+0000","last_peered":"2024-09-16T04:53:07.536856+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:07.536856+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39404137,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10255,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3090},{"shards":"2","objects":6214}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10550","reported_seq":10653,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:07.487867+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:07.487867+0000","last_peered":"2024-09-16T04:53:07.487867+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:07.487867+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40888123,"num_objects":9985,"num_object_clones":0,"num_object_copies":19970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10547,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3234},{"shards":"2","objects":6471}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10287","reported_seq":13794,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:03.460316+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:03.460316+0000","last_peered":"2024-09-16T04:53:03.460316+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:03.460316+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40595456,"num_objects":9911,"num_object_clones":0,"num_object_copies":19822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1636,"num_objects_unfound":0,"num_objects_dirty":9911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7041,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1687,"num_bytes_recovered":6909952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8273},{"shards":"0,2","objects":1636}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'10422","reported_seq":10562,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:03.391386+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:03.391386+0000","last_peered":"2024-09-16T04:53:03.391386+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:03.391386+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40936041,"num_objects":9995,"num_object_clones":0,"num_object_copies":19990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7191,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6471},{"shards":"0,2","objects":3316}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":642498807,"num_objects":156867,"num_object_clones":0,"num_object_copies":313734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1636,"num_objects_unfound":0,"num_objects_dirty":156867,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129763,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5042,"num_bytes_recovered":20619264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1315856384,"data_stored":1315768223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160885,"ondisk_log_size":160885,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":771044,"kb_used_data":472784,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93600796,"statfs":{"total":96636764160,"available":95847215104,"internally_reserved":0,"allocated":484130816,"data_stored":483857761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593771,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594472,"kb_used_data":337880,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93777368,"statfs":{"total":96636764160,"available":96028024832,"internally_reserved":0,"allocated":345989120,"data_stored":345646485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":787884,"kb_used_data":477212,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93583956,"statfs":{"total":96636764160,"available":95829970944,"internally_reserved":0,"allocated":488665088,"data_stored":488396105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":487194624,"data_stored":487191145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":345567232,"data_stored":345490101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":483094528,"data_stored":483086977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:09.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:53:09.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'10070", 'reported_seq': 10150, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:03.427941+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:53:03.427941+0000', 'last_peered': '2024-09-16T04:53:03.427941+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:53:03.427941+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39600128, 'num_objects': 9668, 'num_object_clones': 0, 'num_object_copies': 19336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9668, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10070, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6327}, {'shards': '0,2', 'objects': 3139}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:09.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:53:09.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'10225", 'reported_seq': 10363, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:07.545946+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:53:07.545946+0000', 'last_peered': '2024-09-16T04:53:07.545946+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:53:07.545946+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39452696, 'num_objects': 9633, 'num_object_clones': 0, 'num_object_copies': 19266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9633, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6964, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3025}, {'shards': '2', 'objects': 6314}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:09.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:53:09.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'10324", 'reported_seq': 10458, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:03.427245+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:53:03.427245+0000', 'last_peered': '2024-09-16T04:53:03.427245+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:53:03.427245+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40914944, 'num_objects': 9989, 'num_object_clones': 0, 'num_object_copies': 19978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9989, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7317, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6533}, {'shards': '0,2', 'objects': 3291}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:09.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:53:09.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'10119", 'reported_seq': 10268, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:03.472406+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:53:03.472406+0000', 'last_peered': '2024-09-16T04:53:03.472406+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:53:03.472406+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39899136, 'num_objects': 9741, 'num_object_clones': 0, 'num_object_copies': 19482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9741, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6774, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6342}, {'shards': '0,2', 'objects': 3213}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:09.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:53:09.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'10353", 'reported_seq': 10581, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:07.485320+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:53:07.485320+0000', 'last_peered': '2024-09-16T04:53:07.485320+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:53:07.485320+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39845888, 'num_objects': 9728, 'num_object_clones': 0, 'num_object_copies': 19456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9728, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7104, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3025}, {'shards': '2', 'objects': 6407}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:09.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:53:09.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'10301", 'reported_seq': 10431, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:03.425294+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:53:03.425294+0000', 'last_peered': '2024-09-16T04:53:03.425294+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:53:03.425294+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40620032, 'num_objects': 9917, 'num_object_clones': 0, 'num_object_copies': 19834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9917, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6956, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6452}, {'shards': '0,2', 'objects': 3277}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:09.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:53:09.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'10299", 'reported_seq': 10541, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:07.439983+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:53:07.439983+0000', 'last_peered': '2024-09-16T04:53:07.439983+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:53:07.439983+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39650514, 'num_objects': 9682, 'num_object_clones': 0, 'num_object_copies': 19364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9682, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10271, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3002}, {'shards': '2', 'objects': 6386}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:09.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:53:09.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'10256", 'reported_seq': 10354, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:07.536856+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:53:07.536856+0000', 'last_peered': '2024-09-16T04:53:07.536856+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:53:07.536856+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39404137, 'num_objects': 9621, 'num_object_clones': 0, 'num_object_copies': 19242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9621, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10255, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3090}, {'shards': '2', 'objects': 6214}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:09.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:53:09.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'10550", 'reported_seq': 10653, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:07.487867+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:53:07.487867+0000', 'last_peered': '2024-09-16T04:53:07.487867+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:53:07.487867+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40888123, 'num_objects': 9985, 'num_object_clones': 0, 'num_object_copies': 19970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9985, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10547, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3234}, {'shards': '2', 'objects': 6471}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:09.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:53:09.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'10287", 'reported_seq': 13794, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:03.460316+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:53:03.460316+0000', 'last_peered': '2024-09-16T04:53:03.460316+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:53:03.460316+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40595456, 'num_objects': 9911, 'num_object_clones': 0, 'num_object_copies': 19822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1636, 'num_objects_unfound': 0, 'num_objects_dirty': 9911, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7041, 'num_write_kb': 27420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1687, 'num_bytes_recovered': 6909952, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8273}, {'shards': '0,2', 'objects': 1636}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:09.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:53:09.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'10422", 'reported_seq': 10562, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:03.391386+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:53:03.391386+0000', 'last_peered': '2024-09-16T04:53:03.391386+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:53:03.391386+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40936041, 'num_objects': 9995, 'num_object_clones': 0, 'num_object_copies': 19990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9995, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7191, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6471}, {'shards': '0,2', 'objects': 3316}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:09.109 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:53:10.208 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:10.209 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/313738 objects degraded (10.857%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":337,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156869,"data_bytes":642498826,"bytes_used":2205081600,"bytes_avail":287705210880,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":313738,"degraded_ratio":0.10856829583920341,"misplaced_objects":1636,"misplaced_total":313738,"misplaced_ratio":0.0052145420701349534,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":74733,"recovering_keys_per_sec":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":616042,"read_op_per_sec":0,"write_op_per_sec":469},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:53:10.290 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:53:10.290 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:11.082 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:11.083 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:11.156 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-16T04:53:10.370166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637980938,"num_objects":155766,"num_object_clones":0,"num_object_copies":311532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1512,"num_objects_unfound":0,"num_objects_dirty":155766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130868,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5167,"num_bytes_recovered":21127187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160891,"ondisk_log_size":160891,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2135284,"kb_used_data":1282240,"kb_used_omap":12,"kb_used_meta":852979,"kb_avail":280980236,"statfs":{"total":289910292480,"available":287723761664,"internally_reserved":0,"allocated":1313013760,"data_stored":1312129087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":873450497},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,7,2,9,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16748544,"num_objects":-4089,"num_object_clones":0,"num_object_copies":-8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-343,"num_objects_unfound":0,"num_objects_dirty":-4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5637,"num_write_kb":3096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":343,"num_bytes_recovered":1404928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002970"},"pg_stats":[{"pgid":"2.f","version":"47'10212","reported_seq":10292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:08.417391+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:08.417391+0000","last_peered":"2024-09-16T04:53:08.417391+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:08.417391+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39018496,"num_objects":9526,"num_object_clones":0,"num_object_copies":19052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10212,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6185},{"shards":"0,2","objects":2997}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10295","reported_seq":10411,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:07.478147+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:07.478147+0000","last_peered":"2024-09-16T04:53:07.478147+0000","last_clean":"2024-09-16T04:53:07.478147+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:07.478147+0000","last_undegraded":"2024-09-16T04:53:07.478147+0000","last_fullsized":"2024-09-16T04:53:07.478147+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39727104,"num_objects":9699,"num_object_clones":0,"num_object_copies":19398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7264,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10225","reported_seq":10363,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:07.545946+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:07.545946+0000","last_peered":"2024-09-16T04:53:07.545946+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:07.545946+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39452696,"num_objects":9633,"num_object_clones":0,"num_object_copies":19266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6964,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3025},{"shards":"2","objects":6314}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10454","reported_seq":10524,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:07.493124+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:07.493124+0000","last_peered":"2024-09-16T04:53:07.493124+0000","last_clean":"2024-09-16T04:53:07.493124+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:07.493124+0000","last_undegraded":"2024-09-16T04:53:07.493124+0000","last_fullsized":"2024-09-16T04:53:07.493124+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40509440,"num_objects":9890,"num_object_clones":0,"num_object_copies":19780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10454,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10375","reported_seq":10594,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:08.441258+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:08.441258+0000","last_peered":"2024-09-16T04:53:08.441258+0000","last_clean":"2024-09-16T04:53:08.441258+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:08.441258+0000","last_undegraded":"2024-09-16T04:53:08.441258+0000","last_fullsized":"2024-09-16T04:53:08.441258+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40013824,"num_objects":9769,"num_object_clones":0,"num_object_copies":19538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7407,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10470","reported_seq":10604,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:08.496863+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:08.496863+0000","last_peered":"2024-09-16T04:53:08.496863+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:08.496863+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40316928,"num_objects":9843,"num_object_clones":0,"num_object_copies":19686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7463,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6387},{"shards":"0,2","objects":3145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10254","reported_seq":10403,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:08.366574+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:08.366574+0000","last_peered":"2024-09-16T04:53:08.366574+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:08.366574+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39346176,"num_objects":9606,"num_object_clones":0,"num_object_copies":19212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6909,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6207},{"shards":"0,2","objects":3078}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10310","reported_seq":10526,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:08.344058+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:08.344058+0000","last_peered":"2024-09-16T04:53:08.344058+0000","last_clean":"2024-09-16T04:53:08.344058+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:08.344058+0000","last_undegraded":"2024-09-16T04:53:08.344058+0000","last_fullsized":"2024-09-16T04:53:08.344058+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39755776,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7380,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10353","reported_seq":10581,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:07.485320+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:07.485320+0000","last_peered":"2024-09-16T04:53:07.485320+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:07.485320+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39845888,"num_objects":9728,"num_object_clones":0,"num_object_copies":19456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7104,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3025},{"shards":"2","objects":6407}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10460","reported_seq":10588,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:08.509732+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:08.509732+0000","last_peered":"2024-09-16T04:53:08.509732+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:08.509732+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39968768,"num_objects":9758,"num_object_clones":0,"num_object_copies":19516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7115,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6293},{"shards":"0,2","objects":3118}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10299","reported_seq":10541,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:07.439983+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:07.439983+0000","last_peered":"2024-09-16T04:53:07.439983+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:07.439983+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39650514,"num_objects":9682,"num_object_clones":0,"num_object_copies":19364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10271,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3002},{"shards":"2","objects":6386}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10286","reported_seq":17143,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:07.486667+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:07.486667+0000","last_peered":"2024-09-16T04:53:07.486667+0000","last_clean":"2024-09-16T04:53:07.486667+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:07.486667+0000","last_undegraded":"2024-09-16T04:53:07.486667+0000","last_fullsized":"2024-09-16T04:53:07.486667+0000","mapping_epoch":46,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39698432,"num_objects":9692,"num_object_clones":0,"num_object_copies":19384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7009,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10256","reported_seq":10354,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:07.536856+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:07.536856+0000","last_peered":"2024-09-16T04:53:07.536856+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:07.536856+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39404137,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10255,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3090},{"shards":"2","objects":6214}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10550","reported_seq":10653,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:07.487867+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:07.487867+0000","last_peered":"2024-09-16T04:53:07.487867+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:07.487867+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40888123,"num_objects":9985,"num_object_clones":0,"num_object_copies":19970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10547,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3234},{"shards":"2","objects":6471}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10404","reported_seq":14159,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:08.489348+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:08.489348+0000","last_peered":"2024-09-16T04:53:08.489348+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:08.489348+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40116224,"num_objects":9794,"num_object_clones":0,"num_object_copies":19588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1512,"num_objects_unfound":0,"num_objects_dirty":9794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7158,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1811,"num_bytes_recovered":7417856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8280},{"shards":"0,2","objects":1512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'10585","reported_seq":10725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:08.491575+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:08.491575+0000","last_peered":"2024-09-16T04:53:08.491575+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:08.491575+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40268393,"num_objects":9832,"num_object_clones":0,"num_object_copies":19664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7354,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6308},{"shards":"0,2","objects":3153}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":637980919,"num_objects":155764,"num_object_clones":0,"num_object_copies":311528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1512,"num_objects_unfound":0,"num_objects_dirty":155764,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130866,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5166,"num_bytes_recovered":21127168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1310195712,"data_stored":1310107551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160888,"ondisk_log_size":160888,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":771044,"kb_used_data":472784,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93600796,"statfs":{"total":96636764160,"available":95847215104,"internally_reserved":0,"allocated":484130816,"data_stored":483857761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593771,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594472,"kb_used_data":337880,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93777368,"statfs":{"total":96636764160,"available":96028024832,"internally_reserved":0,"allocated":345989120,"data_stored":345646485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769768,"kb_used_data":471576,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93602072,"statfs":{"total":96636764160,"available":95848521728,"internally_reserved":0,"allocated":482893824,"data_stored":482624841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,3,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":481533952,"data_stored":481530473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":345567232,"data_stored":345490101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":483094528,"data_stored":483086977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:14.159 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:14.994 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:15.040 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-16T04:53:14.371160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":628637962,"num_objects":153485,"num_object_clones":0,"num_object_copies":306970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1387,"num_objects_unfound":0,"num_objects_dirty":153485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":133149,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5292,"num_bytes_recovered":21639187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160672,"ondisk_log_size":160672,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2141680,"kb_used_data":1270588,"kb_used_omap":12,"kb_used_meta":871027,"kb_avail":280973840,"statfs":{"total":289910292480,"available":287717212160,"internally_reserved":0,"allocated":1301082112,"data_stored":1300197439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":891931649},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,7,4,4,4,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28033024,"num_objects":-6844,"num_object_clones":0,"num_object_copies":-13688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-363,"num_objects_unfound":0,"num_objects_dirty":-6844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6844,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":363,"num_bytes_recovered":1486848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002940"},"pg_stats":[{"pgid":"2.f","version":"47'10330","reported_seq":10410,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.483172+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:13.483172+0000","last_peered":"2024-09-16T04:53:13.483172+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:13.483172+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38535168,"num_objects":9408,"num_object_clones":0,"num_object_copies":18816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10330,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6067},{"shards":"0,2","objects":2879}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10434","reported_seq":10548,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:12.598668+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:12.598668+0000","last_peered":"2024-09-16T04:53:12.598668+0000","last_clean":"2024-09-16T04:53:12.598668+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:12.598668+0000","last_undegraded":"2024-09-16T04:53:12.598668+0000","last_fullsized":"2024-09-16T04:53:12.598668+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39157760,"num_objects":9560,"num_object_clones":0,"num_object_copies":19120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7403,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10347","reported_seq":10484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.611849+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:12.611849+0000","last_peered":"2024-09-16T04:53:12.611849+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:12.611849+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38952984,"num_objects":9511,"num_object_clones":0,"num_object_copies":19022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7086,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2903},{"shards":"2","objects":6192}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10606","reported_seq":10675,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:12.561431+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:12.561431+0000","last_peered":"2024-09-16T04:53:12.561431+0000","last_clean":"2024-09-16T04:53:12.561431+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:12.561431+0000","last_undegraded":"2024-09-16T04:53:12.561431+0000","last_fullsized":"2024-09-16T04:53:12.561431+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39886848,"num_objects":9738,"num_object_clones":0,"num_object_copies":19476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10606,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10540","reported_seq":10759,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:13.528693+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:13.528693+0000","last_peered":"2024-09-16T04:53:13.528693+0000","last_clean":"2024-09-16T04:53:13.528693+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:13.528693+0000","last_undegraded":"2024-09-16T04:53:13.528693+0000","last_fullsized":"2024-09-16T04:53:13.528693+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7572,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10609","reported_seq":10743,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.461114+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:13.461114+0000","last_peered":"2024-09-16T04:53:13.461114+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:13.461114+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39747584,"num_objects":9704,"num_object_clones":0,"num_object_copies":19408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7602,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6248},{"shards":"0,2","objects":3006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10414","reported_seq":10560,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.480290+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:13.480290+0000","last_peered":"2024-09-16T04:53:13.480290+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:13.480290+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38690816,"num_objects":9446,"num_object_clones":0,"num_object_copies":18892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6047},{"shards":"0,2","objects":2918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10442","reported_seq":10658,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:13.542375+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:13.542375+0000","last_peered":"2024-09-16T04:53:13.542375+0000","last_clean":"2024-09-16T04:53:13.542375+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:13.542375+0000","last_undegraded":"2024-09-16T04:53:13.542375+0000","last_fullsized":"2024-09-16T04:53:13.542375+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215104,"num_objects":9574,"num_object_clones":0,"num_object_copies":19148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7512,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10499","reported_seq":10727,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.570678+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:12.570678+0000","last_peered":"2024-09-16T04:53:12.570678+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:12.570678+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39247872,"num_objects":9582,"num_object_clones":0,"num_object_copies":19164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7250,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2879},{"shards":"2","objects":6261}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10610","reported_seq":10740,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.532520+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:13.532520+0000","last_peered":"2024-09-16T04:53:13.532520+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:13.532520+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39354368,"num_objects":9608,"num_object_clones":0,"num_object_copies":19216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7265,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6143},{"shards":"0,2","objects":2968}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10423","reported_seq":10665,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.582645+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:12.582645+0000","last_peered":"2024-09-16T04:53:12.582645+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:12.582645+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39142610,"num_objects":9558,"num_object_clones":0,"num_object_copies":19116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10395,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2878},{"shards":"2","objects":6262}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10418","reported_seq":17275,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:12.583986+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:12.583986+0000","last_peered":"2024-09-16T04:53:12.583986+0000","last_clean":"2024-09-16T04:53:12.583986+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:12.583986+0000","last_undegraded":"2024-09-16T04:53:12.583986+0000","last_fullsized":"2024-09-16T04:53:12.583986+0000","mapping_epoch":46,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39157760,"num_objects":9560,"num_object_clones":0,"num_object_copies":19120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7141,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10400","reported_seq":10498,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.574801+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:12.574801+0000","last_peered":"2024-09-16T04:53:12.574801+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:12.574801+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38814313,"num_objects":9477,"num_object_clones":0,"num_object_copies":18954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10399,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2946},{"shards":"2","objects":6070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10698","reported_seq":10800,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.598972+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:12.598972+0000","last_peered":"2024-09-16T04:53:12.598972+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:12.598972+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40281915,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10695,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3086},{"shards":"2","objects":6323}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10532","reported_seq":14535,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.544647+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:13.544647+0000","last_peered":"2024-09-16T04:53:13.544647+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:13.544647+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39591936,"num_objects":9666,"num_object_clones":0,"num_object_copies":19332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1387,"num_objects_unfound":0,"num_objects_dirty":9666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1936,"num_bytes_recovered":7929856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8277},{"shards":"0,2","objects":1387}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'10767","reported_seq":10906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.496548+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:13.496548+0000","last_peered":"2024-09-16T04:53:13.496548+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:13.496548+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39522921,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7536,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6126},{"shards":"0,2","objects":2971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":628637943,"num_objects":153483,"num_object_clones":0,"num_object_copies":306966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1387,"num_objects_unfound":0,"num_objects_dirty":153483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":133147,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5291,"num_bytes_recovered":21639168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1297952768,"data_stored":1297864607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160669,"ondisk_log_size":160669,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769796,"kb_used_data":467376,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93602044,"statfs":{"total":96636764160,"available":95848493056,"internally_reserved":0,"allocated":478593024,"data_stored":478319969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593772,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603616,"kb_used_data":337296,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93768224,"statfs":{"total":96636764160,"available":96018661376,"internally_reserved":0,"allocated":345391104,"data_stored":345048469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":768268,"kb_used_data":465916,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93603572,"statfs":{"total":96636764160,"available":95850057728,"internally_reserved":0,"allocated":477097984,"data_stored":476829001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6,2,2,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":475529216,"data_stored":475525737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":344977408,"data_stored":344900277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":477446144,"data_stored":477438593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:15.042 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:15.981 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:15.982 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:16.026 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-16T04:53:14.371160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":628637962,"num_objects":153485,"num_object_clones":0,"num_object_copies":306970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1387,"num_objects_unfound":0,"num_objects_dirty":153485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":133149,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5292,"num_bytes_recovered":21639187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160672,"ondisk_log_size":160672,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2141680,"kb_used_data":1270588,"kb_used_omap":12,"kb_used_meta":871027,"kb_avail":280973840,"statfs":{"total":289910292480,"available":287717212160,"internally_reserved":0,"allocated":1301082112,"data_stored":1300197439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":891931649},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,7,4,4,4,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28033024,"num_objects":-6844,"num_object_clones":0,"num_object_copies":-13688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-363,"num_objects_unfound":0,"num_objects_dirty":-6844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6844,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":363,"num_bytes_recovered":1486848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002940"},"pg_stats":[{"pgid":"2.f","version":"47'10330","reported_seq":10410,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.483172+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:13.483172+0000","last_peered":"2024-09-16T04:53:13.483172+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:13.483172+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38535168,"num_objects":9408,"num_object_clones":0,"num_object_copies":18816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10330,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6067},{"shards":"0,2","objects":2879}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10434","reported_seq":10548,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:12.598668+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:12.598668+0000","last_peered":"2024-09-16T04:53:12.598668+0000","last_clean":"2024-09-16T04:53:12.598668+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:12.598668+0000","last_undegraded":"2024-09-16T04:53:12.598668+0000","last_fullsized":"2024-09-16T04:53:12.598668+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39157760,"num_objects":9560,"num_object_clones":0,"num_object_copies":19120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7403,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10347","reported_seq":10484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.611849+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:12.611849+0000","last_peered":"2024-09-16T04:53:12.611849+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:12.611849+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38952984,"num_objects":9511,"num_object_clones":0,"num_object_copies":19022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7086,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2903},{"shards":"2","objects":6192}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10606","reported_seq":10675,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:12.561431+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:12.561431+0000","last_peered":"2024-09-16T04:53:12.561431+0000","last_clean":"2024-09-16T04:53:12.561431+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:12.561431+0000","last_undegraded":"2024-09-16T04:53:12.561431+0000","last_fullsized":"2024-09-16T04:53:12.561431+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39886848,"num_objects":9738,"num_object_clones":0,"num_object_copies":19476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10606,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10540","reported_seq":10759,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:13.528693+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:13.528693+0000","last_peered":"2024-09-16T04:53:13.528693+0000","last_clean":"2024-09-16T04:53:13.528693+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:13.528693+0000","last_undegraded":"2024-09-16T04:53:13.528693+0000","last_fullsized":"2024-09-16T04:53:13.528693+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7572,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10609","reported_seq":10743,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.461114+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:13.461114+0000","last_peered":"2024-09-16T04:53:13.461114+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:13.461114+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39747584,"num_objects":9704,"num_object_clones":0,"num_object_copies":19408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7602,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6248},{"shards":"0,2","objects":3006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10414","reported_seq":10560,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.480290+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:13.480290+0000","last_peered":"2024-09-16T04:53:13.480290+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:13.480290+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38690816,"num_objects":9446,"num_object_clones":0,"num_object_copies":18892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6047},{"shards":"0,2","objects":2918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10442","reported_seq":10658,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:13.542375+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:13.542375+0000","last_peered":"2024-09-16T04:53:13.542375+0000","last_clean":"2024-09-16T04:53:13.542375+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:13.542375+0000","last_undegraded":"2024-09-16T04:53:13.542375+0000","last_fullsized":"2024-09-16T04:53:13.542375+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215104,"num_objects":9574,"num_object_clones":0,"num_object_copies":19148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7512,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10499","reported_seq":10727,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.570678+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:12.570678+0000","last_peered":"2024-09-16T04:53:12.570678+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:12.570678+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39247872,"num_objects":9582,"num_object_clones":0,"num_object_copies":19164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7250,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2879},{"shards":"2","objects":6261}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10610","reported_seq":10740,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.532520+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:13.532520+0000","last_peered":"2024-09-16T04:53:13.532520+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:13.532520+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39354368,"num_objects":9608,"num_object_clones":0,"num_object_copies":19216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7265,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6143},{"shards":"0,2","objects":2968}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10423","reported_seq":10665,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.582645+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:12.582645+0000","last_peered":"2024-09-16T04:53:12.582645+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:12.582645+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39142610,"num_objects":9558,"num_object_clones":0,"num_object_copies":19116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10395,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2878},{"shards":"2","objects":6262}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10418","reported_seq":17275,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:12.583986+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:12.583986+0000","last_peered":"2024-09-16T04:53:12.583986+0000","last_clean":"2024-09-16T04:53:12.583986+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:12.583986+0000","last_undegraded":"2024-09-16T04:53:12.583986+0000","last_fullsized":"2024-09-16T04:53:12.583986+0000","mapping_epoch":46,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39157760,"num_objects":9560,"num_object_clones":0,"num_object_copies":19120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7141,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10400","reported_seq":10498,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.574801+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:12.574801+0000","last_peered":"2024-09-16T04:53:12.574801+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:12.574801+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38814313,"num_objects":9477,"num_object_clones":0,"num_object_copies":18954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10399,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2946},{"shards":"2","objects":6070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10698","reported_seq":10800,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.598972+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:12.598972+0000","last_peered":"2024-09-16T04:53:12.598972+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:12.598972+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40281915,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10695,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3086},{"shards":"2","objects":6323}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10532","reported_seq":14535,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.544647+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:13.544647+0000","last_peered":"2024-09-16T04:53:13.544647+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:13.544647+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39591936,"num_objects":9666,"num_object_clones":0,"num_object_copies":19332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1387,"num_objects_unfound":0,"num_objects_dirty":9666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1936,"num_bytes_recovered":7929856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8277},{"shards":"0,2","objects":1387}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'10767","reported_seq":10906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.496548+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:13.496548+0000","last_peered":"2024-09-16T04:53:13.496548+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:13.496548+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39522921,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7536,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6126},{"shards":"0,2","objects":2971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":628637943,"num_objects":153483,"num_object_clones":0,"num_object_copies":306966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1387,"num_objects_unfound":0,"num_objects_dirty":153483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":133147,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5291,"num_bytes_recovered":21639168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1297952768,"data_stored":1297864607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160669,"ondisk_log_size":160669,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769796,"kb_used_data":467376,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93602044,"statfs":{"total":96636764160,"available":95848493056,"internally_reserved":0,"allocated":478593024,"data_stored":478319969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593772,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603616,"kb_used_data":337296,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93768224,"statfs":{"total":96636764160,"available":96018661376,"internally_reserved":0,"allocated":345391104,"data_stored":345048469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":768268,"kb_used_data":465916,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93603572,"statfs":{"total":96636764160,"available":95850057728,"internally_reserved":0,"allocated":477097984,"data_stored":476829001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6,2,2,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":475529216,"data_stored":475525737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":344977408,"data_stored":344900277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":477446144,"data_stored":477438593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:16.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:53:16.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'10330", 'reported_seq': 10410, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:13.483172+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:53:13.483172+0000', 'last_peered': '2024-09-16T04:53:13.483172+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:53:13.483172+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38535168, 'num_objects': 9408, 'num_object_clones': 0, 'num_object_copies': 18816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9408, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10330, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6067}, {'shards': '0,2', 'objects': 2879}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:16.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:53:16.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'10347", 'reported_seq': 10484, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:12.611849+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:53:12.611849+0000', 'last_peered': '2024-09-16T04:53:12.611849+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:53:12.611849+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38952984, 'num_objects': 9511, 'num_object_clones': 0, 'num_object_copies': 19022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9511, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7086, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2903}, {'shards': '2', 'objects': 6192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:16.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:53:16.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'10609", 'reported_seq': 10743, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:13.461114+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:53:13.461114+0000', 'last_peered': '2024-09-16T04:53:13.461114+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:53:13.461114+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39747584, 'num_objects': 9704, 'num_object_clones': 0, 'num_object_copies': 19408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9704, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7602, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6248}, {'shards': '0,2', 'objects': 3006}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:16.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:53:16.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'10414", 'reported_seq': 10560, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:13.480290+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:53:13.480290+0000', 'last_peered': '2024-09-16T04:53:13.480290+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:53:13.480290+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38690816, 'num_objects': 9446, 'num_object_clones': 0, 'num_object_copies': 18892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9446, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7069, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6047}, {'shards': '0,2', 'objects': 2918}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:16.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:53:16.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'10499", 'reported_seq': 10727, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:12.570678+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:53:12.570678+0000', 'last_peered': '2024-09-16T04:53:12.570678+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:53:12.570678+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39247872, 'num_objects': 9582, 'num_object_clones': 0, 'num_object_copies': 19164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9582, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7250, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2879}, {'shards': '2', 'objects': 6261}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:16.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:53:16.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'10610", 'reported_seq': 10740, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:13.532520+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:53:13.532520+0000', 'last_peered': '2024-09-16T04:53:13.532520+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:53:13.532520+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39354368, 'num_objects': 9608, 'num_object_clones': 0, 'num_object_copies': 19216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9608, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7265, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6143}, {'shards': '0,2', 'objects': 2968}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:16.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:53:16.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'10423", 'reported_seq': 10665, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:12.582645+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:53:12.582645+0000', 'last_peered': '2024-09-16T04:53:12.582645+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:53:12.582645+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39142610, 'num_objects': 9558, 'num_object_clones': 0, 'num_object_copies': 19116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9558, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10395, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2878}, {'shards': '2', 'objects': 6262}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:16.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:53:16.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'10400", 'reported_seq': 10498, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:12.574801+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:53:12.574801+0000', 'last_peered': '2024-09-16T04:53:12.574801+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:53:12.574801+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38814313, 'num_objects': 9477, 'num_object_clones': 0, 'num_object_copies': 18954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10399, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2946}, {'shards': '2', 'objects': 6070}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:16.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:53:16.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'10698", 'reported_seq': 10800, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:12.598972+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:53:12.598972+0000', 'last_peered': '2024-09-16T04:53:12.598972+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:53:12.598972+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40281915, 'num_objects': 9837, 'num_object_clones': 0, 'num_object_copies': 19674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9837, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10695, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3086}, {'shards': '2', 'objects': 6323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:16.029 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:53:16.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'10532", 'reported_seq': 14535, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:13.544647+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:53:13.544647+0000', 'last_peered': '2024-09-16T04:53:13.544647+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:53:13.544647+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39591936, 'num_objects': 9666, 'num_object_clones': 0, 'num_object_copies': 19332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1387, 'num_objects_unfound': 0, 'num_objects_dirty': 9666, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7286, 'num_write_kb': 27420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1936, 'num_bytes_recovered': 7929856, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8277}, {'shards': '0,2', 'objects': 1387}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:16.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:53:16.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'10767", 'reported_seq': 10906, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:13.496548+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:53:13.496548+0000', 'last_peered': '2024-09-16T04:53:13.496548+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:53:13.496548+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39522921, 'num_objects': 9650, 'num_object_clones': 0, 'num_object_copies': 19300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7536, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6126}, {'shards': '0,2', 'objects': 2971}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:16.030 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:53:16.935 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:16.935 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/306970 objects degraded (11.096%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":344,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":153485,"data_bytes":628637962,"bytes_used":2193080320,"bytes_avail":287717212160,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":306970,"degraded_ratio":0.11096198325569274,"misplaced_objects":1387,"misplaced_total":306970,"misplaced_ratio":0.0045183568426882103,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":84970,"recovering_keys_per_sec":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":379},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:53:17.008 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:53:17.008 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:17.847 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:17.848 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:17.900 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-16T04:53:16.371698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":628637962,"num_objects":153485,"num_object_clones":0,"num_object_copies":306970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1387,"num_objects_unfound":0,"num_objects_dirty":153485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":133149,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5292,"num_bytes_recovered":21639187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160672,"ondisk_log_size":160672,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2141680,"kb_used_data":1270588,"kb_used_omap":12,"kb_used_meta":871027,"kb_avail":280973840,"statfs":{"total":289910292480,"available":287717212160,"internally_reserved":0,"allocated":1301082112,"data_stored":1300197439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":891931649},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,7,4,4,4,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18677760,"num_objects":-4560,"num_object_clones":0,"num_object_copies":-9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":-4560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4560,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002994"},"pg_stats":[{"pgid":"2.f","version":"47'10330","reported_seq":10410,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.483172+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:13.483172+0000","last_peered":"2024-09-16T04:53:13.483172+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:13.483172+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38535168,"num_objects":9408,"num_object_clones":0,"num_object_copies":18816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10330,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6067},{"shards":"0,2","objects":2879}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10434","reported_seq":10548,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:12.598668+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:12.598668+0000","last_peered":"2024-09-16T04:53:12.598668+0000","last_clean":"2024-09-16T04:53:12.598668+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:12.598668+0000","last_undegraded":"2024-09-16T04:53:12.598668+0000","last_fullsized":"2024-09-16T04:53:12.598668+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39157760,"num_objects":9560,"num_object_clones":0,"num_object_copies":19120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7403,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10347","reported_seq":10484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.611849+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:12.611849+0000","last_peered":"2024-09-16T04:53:12.611849+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:12.611849+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38952984,"num_objects":9511,"num_object_clones":0,"num_object_copies":19022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7086,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2903},{"shards":"2","objects":6192}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10606","reported_seq":10675,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:12.561431+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:12.561431+0000","last_peered":"2024-09-16T04:53:12.561431+0000","last_clean":"2024-09-16T04:53:12.561431+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:12.561431+0000","last_undegraded":"2024-09-16T04:53:12.561431+0000","last_fullsized":"2024-09-16T04:53:12.561431+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39886848,"num_objects":9738,"num_object_clones":0,"num_object_copies":19476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10606,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10540","reported_seq":10759,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:13.528693+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:13.528693+0000","last_peered":"2024-09-16T04:53:13.528693+0000","last_clean":"2024-09-16T04:53:13.528693+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:13.528693+0000","last_undegraded":"2024-09-16T04:53:13.528693+0000","last_fullsized":"2024-09-16T04:53:13.528693+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39337984,"num_objects":9604,"num_object_clones":0,"num_object_copies":19208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7572,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10609","reported_seq":10743,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.461114+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:13.461114+0000","last_peered":"2024-09-16T04:53:13.461114+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:13.461114+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39747584,"num_objects":9704,"num_object_clones":0,"num_object_copies":19408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7602,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6248},{"shards":"0,2","objects":3006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10414","reported_seq":10560,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.480290+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:13.480290+0000","last_peered":"2024-09-16T04:53:13.480290+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:13.480290+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38690816,"num_objects":9446,"num_object_clones":0,"num_object_copies":18892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6047},{"shards":"0,2","objects":2918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10442","reported_seq":10658,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:13.542375+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:13.542375+0000","last_peered":"2024-09-16T04:53:13.542375+0000","last_clean":"2024-09-16T04:53:13.542375+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:13.542375+0000","last_undegraded":"2024-09-16T04:53:13.542375+0000","last_fullsized":"2024-09-16T04:53:13.542375+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39215104,"num_objects":9574,"num_object_clones":0,"num_object_copies":19148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7512,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10499","reported_seq":10727,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.570678+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:12.570678+0000","last_peered":"2024-09-16T04:53:12.570678+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:12.570678+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39247872,"num_objects":9582,"num_object_clones":0,"num_object_copies":19164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7250,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2879},{"shards":"2","objects":6261}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10610","reported_seq":10740,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.532520+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:13.532520+0000","last_peered":"2024-09-16T04:53:13.532520+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:13.532520+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39354368,"num_objects":9608,"num_object_clones":0,"num_object_copies":19216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7265,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6143},{"shards":"0,2","objects":2968}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10423","reported_seq":10665,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.582645+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:12.582645+0000","last_peered":"2024-09-16T04:53:12.582645+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:12.582645+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39142610,"num_objects":9558,"num_object_clones":0,"num_object_copies":19116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10395,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2878},{"shards":"2","objects":6262}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10418","reported_seq":17275,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:12.583986+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:12.583986+0000","last_peered":"2024-09-16T04:53:12.583986+0000","last_clean":"2024-09-16T04:53:12.583986+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:12.583986+0000","last_undegraded":"2024-09-16T04:53:12.583986+0000","last_fullsized":"2024-09-16T04:53:12.583986+0000","mapping_epoch":46,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39157760,"num_objects":9560,"num_object_clones":0,"num_object_copies":19120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7141,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10400","reported_seq":10498,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.574801+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:12.574801+0000","last_peered":"2024-09-16T04:53:12.574801+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:12.574801+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38814313,"num_objects":9477,"num_object_clones":0,"num_object_copies":18954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10399,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2946},{"shards":"2","objects":6070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10698","reported_seq":10800,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:12.598972+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:12.598972+0000","last_peered":"2024-09-16T04:53:12.598972+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:12.598972+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40281915,"num_objects":9837,"num_object_clones":0,"num_object_copies":19674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10695,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3086},{"shards":"2","objects":6323}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10532","reported_seq":14535,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.544647+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:13.544647+0000","last_peered":"2024-09-16T04:53:13.544647+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:13.544647+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39591936,"num_objects":9666,"num_object_clones":0,"num_object_copies":19332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1387,"num_objects_unfound":0,"num_objects_dirty":9666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1936,"num_bytes_recovered":7929856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8277},{"shards":"0,2","objects":1387}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'10767","reported_seq":10906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:13.496548+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:13.496548+0000","last_peered":"2024-09-16T04:53:13.496548+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:13.496548+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39522921,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7536,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6126},{"shards":"0,2","objects":2971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":628637943,"num_objects":153483,"num_object_clones":0,"num_object_copies":306966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1387,"num_objects_unfound":0,"num_objects_dirty":153483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":133147,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5291,"num_bytes_recovered":21639168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1297952768,"data_stored":1297864607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160669,"ondisk_log_size":160669,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769796,"kb_used_data":467376,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93602044,"statfs":{"total":96636764160,"available":95848493056,"internally_reserved":0,"allocated":478593024,"data_stored":478319969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593772,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603616,"kb_used_data":337296,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93768224,"statfs":{"total":96636764160,"available":96018661376,"internally_reserved":0,"allocated":345391104,"data_stored":345048469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":768268,"kb_used_data":465916,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93603572,"statfs":{"total":96636764160,"available":95850057728,"internally_reserved":0,"allocated":477097984,"data_stored":476829001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6,2,2,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":475529216,"data_stored":475525737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":344977408,"data_stored":344900277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":477446144,"data_stored":477438593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:20.902 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:21.763 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:21.763 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:21.843 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-16T04:53:20.372699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":619507978,"num_objects":151256,"num_object_clones":0,"num_object_copies":302512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":151256,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":135378,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5413,"num_bytes_recovered":22134803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160901,"ondisk_log_size":160901,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2146520,"kb_used_data":1258780,"kb_used_omap":12,"kb_used_meta":887667,"kb_avail":280969000,"statfs":{"total":289910292480,"available":287712256000,"internally_reserved":0,"allocated":1288990720,"data_stored":1288106047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":908971009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,4,1,11,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22990848,"num_objects":-5613,"num_object_clones":0,"num_object_copies":-11226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-370,"num_objects_unfound":0,"num_objects_dirty":-5613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5613,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":370,"num_bytes_recovered":1515520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003027"},"pg_stats":[{"pgid":"2.f","version":"47'10479","reported_seq":10559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:18.602478+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:18.602478+0000","last_peered":"2024-09-16T04:53:18.602478+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:18.602478+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37924864,"num_objects":9259,"num_object_clones":0,"num_object_copies":18518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10479,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5918},{"shards":"0,2","objects":2730}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10574","reported_seq":10691,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:17.637435+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:17.637435+0000","last_peered":"2024-09-16T04:53:17.637435+0000","last_clean":"2024-09-16T04:53:17.637435+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:17.637435+0000","last_undegraded":"2024-09-16T04:53:17.637435+0000","last_fullsized":"2024-09-16T04:53:17.637435+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38584320,"num_objects":9420,"num_object_clones":0,"num_object_copies":18840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7543,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10465","reported_seq":10603,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:17.617461+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:17.617461+0000","last_peered":"2024-09-16T04:53:17.617461+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:17.617461+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38469656,"num_objects":9393,"num_object_clones":0,"num_object_copies":18786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7204,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2785},{"shards":"2","objects":6074}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10724","reported_seq":10793,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:17.575006+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:17.575006+0000","last_peered":"2024-09-16T04:53:17.575006+0000","last_clean":"2024-09-16T04:53:17.575006+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:17.575006+0000","last_undegraded":"2024-09-16T04:53:17.575006+0000","last_fullsized":"2024-09-16T04:53:17.575006+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39403520,"num_objects":9620,"num_object_clones":0,"num_object_copies":19240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10724,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10680","reported_seq":10899,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:18.472723+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:18.472723+0000","last_peered":"2024-09-16T04:53:18.472723+0000","last_clean":"2024-09-16T04:53:18.472723+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:18.472723+0000","last_undegraded":"2024-09-16T04:53:18.472723+0000","last_fullsized":"2024-09-16T04:53:18.472723+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38764544,"num_objects":9464,"num_object_clones":0,"num_object_copies":18928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7712,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10747","reported_seq":10881,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:18.497761+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:18.497761+0000","last_peered":"2024-09-16T04:53:18.497761+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:18.497761+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39182336,"num_objects":9566,"num_object_clones":0,"num_object_copies":19132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7740,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6110},{"shards":"0,2","objects":2868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10563","reported_seq":10712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:18.561820+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:18.561820+0000","last_peered":"2024-09-16T04:53:18.561820+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:18.561820+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38080512,"num_objects":9297,"num_object_clones":0,"num_object_copies":18594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7218,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5898},{"shards":"0,2","objects":2769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10600","reported_seq":10816,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:18.603998+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:18.603998+0000","last_peered":"2024-09-16T04:53:18.603998+0000","last_clean":"2024-09-16T04:53:18.603998+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:18.603998+0000","last_undegraded":"2024-09-16T04:53:18.603998+0000","last_fullsized":"2024-09-16T04:53:18.603998+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38567936,"num_objects":9416,"num_object_clones":0,"num_object_copies":18832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7670,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10638","reported_seq":10866,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:17.613148+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:17.613148+0000","last_peered":"2024-09-16T04:53:17.613148+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:17.613148+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38678528,"num_objects":9443,"num_object_clones":0,"num_object_copies":18886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7389,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2740},{"shards":"2","objects":6122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10759","reported_seq":10889,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:18.454770+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:18.454770+0000","last_peered":"2024-09-16T04:53:18.454770+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:18.454770+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38744064,"num_objects":9459,"num_object_clones":0,"num_object_copies":18918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7414,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5994},{"shards":"0,2","objects":2819}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10566","reported_seq":10808,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:17.614302+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:17.614302+0000","last_peered":"2024-09-16T04:53:17.614302+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:17.614302+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38556882,"num_objects":9415,"num_object_clones":0,"num_object_copies":18830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10538,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2735},{"shards":"2","objects":6119}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10567","reported_seq":17424,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:17.593726+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:17.593726+0000","last_peered":"2024-09-16T04:53:17.593726+0000","last_clean":"2024-09-16T04:53:17.593726+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:17.593726+0000","last_undegraded":"2024-09-16T04:53:17.593726+0000","last_fullsized":"2024-09-16T04:53:17.593726+0000","mapping_epoch":46,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38547456,"num_objects":9411,"num_object_clones":0,"num_object_copies":18822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7290,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10516","reported_seq":10613,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:17.617310+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:17.617310+0000","last_peered":"2024-09-16T04:53:17.617310+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:17.617310+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38339177,"num_objects":9361,"num_object_clones":0,"num_object_copies":18722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10515,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2830},{"shards":"2","objects":5954}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10836","reported_seq":10939,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:17.573299+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:17.573299+0000","last_peered":"2024-09-16T04:53:17.573299+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:17.573299+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39716667,"num_objects":9699,"num_object_clones":0,"num_object_copies":19398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10833,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2948},{"shards":"2","objects":6185}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10670","reported_seq":14917,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:18.601119+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:18.601119+0000","last_peered":"2024-09-16T04:53:18.601119+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:18.601119+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39026688,"num_objects":9528,"num_object_clones":0,"num_object_copies":19056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":9528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7424,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2057,"num_bytes_recovered":8425472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8260},{"shards":"0,2","objects":1266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'10914","reported_seq":11054,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:18.583023+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:18.583023+0000","last_peered":"2024-09-16T04:53:18.583023+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:18.583023+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38920809,"num_objects":9503,"num_object_clones":0,"num_object_copies":19006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7683,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5979},{"shards":"0,2","objects":2824}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":619507959,"num_objects":151254,"num_object_clones":0,"num_object_copies":302508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":151254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":135376,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5412,"num_bytes_recovered":22134784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1285914624,"data_stored":1285826463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":772584,"kb_used_data":461840,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93599256,"statfs":{"total":96636764160,"available":95845638144,"internally_reserved":0,"allocated":472924160,"data_stored":472651105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593773,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607164,"kb_used_data":336684,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93764676,"statfs":{"total":96636764160,"available":96015028224,"internally_reserved":0,"allocated":344764416,"data_stored":344421781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182464,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":766772,"kb_used_data":460256,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93605068,"statfs":{"total":96636764160,"available":95851589632,"internally_reserved":0,"allocated":471302144,"data_stored":471033161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":469684224,"data_stored":469680745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":344342528,"data_stored":344265397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471887872,"data_stored":471880321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:21.845 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:22.752 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:22.755 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:22.814 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-16T04:53:22.373180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":619507978,"num_objects":151256,"num_object_clones":0,"num_object_copies":302512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":151256,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":135378,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5413,"num_bytes_recovered":22134803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160901,"ondisk_log_size":160901,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2137520,"kb_used_data":1258100,"kb_used_omap":12,"kb_used_meta":879347,"kb_avail":280978000,"statfs":{"total":289910292480,"available":287721472000,"internally_reserved":0,"allocated":1288294400,"data_stored":1287416685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":900451329},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,3,6,10,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18472960,"num_objects":-4510,"num_object_clones":0,"num_object_copies":-9020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-246,"num_objects_unfound":0,"num_objects_dirty":-4510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4510,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1007616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003014"},"pg_stats":[{"pgid":"2.f","version":"47'10479","reported_seq":10559,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:18.602478+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:18.602478+0000","last_peered":"2024-09-16T04:53:18.602478+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:18.602478+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37924864,"num_objects":9259,"num_object_clones":0,"num_object_copies":18518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10479,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5918},{"shards":"0,2","objects":2730}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10574","reported_seq":10691,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:17.637435+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:17.637435+0000","last_peered":"2024-09-16T04:53:17.637435+0000","last_clean":"2024-09-16T04:53:17.637435+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:17.637435+0000","last_undegraded":"2024-09-16T04:53:17.637435+0000","last_fullsized":"2024-09-16T04:53:17.637435+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38584320,"num_objects":9420,"num_object_clones":0,"num_object_copies":18840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7543,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10465","reported_seq":10603,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:17.617461+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:17.617461+0000","last_peered":"2024-09-16T04:53:17.617461+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:17.617461+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38469656,"num_objects":9393,"num_object_clones":0,"num_object_copies":18786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7204,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2785},{"shards":"2","objects":6074}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10724","reported_seq":10793,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:17.575006+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:17.575006+0000","last_peered":"2024-09-16T04:53:17.575006+0000","last_clean":"2024-09-16T04:53:17.575006+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:17.575006+0000","last_undegraded":"2024-09-16T04:53:17.575006+0000","last_fullsized":"2024-09-16T04:53:17.575006+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39403520,"num_objects":9620,"num_object_clones":0,"num_object_copies":19240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10724,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10680","reported_seq":10899,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:18.472723+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:18.472723+0000","last_peered":"2024-09-16T04:53:18.472723+0000","last_clean":"2024-09-16T04:53:18.472723+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:18.472723+0000","last_undegraded":"2024-09-16T04:53:18.472723+0000","last_fullsized":"2024-09-16T04:53:18.472723+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38764544,"num_objects":9464,"num_object_clones":0,"num_object_copies":18928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7712,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10747","reported_seq":10881,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:18.497761+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:18.497761+0000","last_peered":"2024-09-16T04:53:18.497761+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:18.497761+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39182336,"num_objects":9566,"num_object_clones":0,"num_object_copies":19132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7740,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6110},{"shards":"0,2","objects":2868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10563","reported_seq":10712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:18.561820+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:18.561820+0000","last_peered":"2024-09-16T04:53:18.561820+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:18.561820+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38080512,"num_objects":9297,"num_object_clones":0,"num_object_copies":18594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7218,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5898},{"shards":"0,2","objects":2769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10600","reported_seq":10816,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:18.603998+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:18.603998+0000","last_peered":"2024-09-16T04:53:18.603998+0000","last_clean":"2024-09-16T04:53:18.603998+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:18.603998+0000","last_undegraded":"2024-09-16T04:53:18.603998+0000","last_fullsized":"2024-09-16T04:53:18.603998+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38567936,"num_objects":9416,"num_object_clones":0,"num_object_copies":18832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7670,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10638","reported_seq":10866,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:17.613148+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:17.613148+0000","last_peered":"2024-09-16T04:53:17.613148+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:17.613148+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38678528,"num_objects":9443,"num_object_clones":0,"num_object_copies":18886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7389,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2740},{"shards":"2","objects":6122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10759","reported_seq":10889,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:18.454770+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:18.454770+0000","last_peered":"2024-09-16T04:53:18.454770+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:18.454770+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38744064,"num_objects":9459,"num_object_clones":0,"num_object_copies":18918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7414,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5994},{"shards":"0,2","objects":2819}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10566","reported_seq":10808,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:17.614302+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:17.614302+0000","last_peered":"2024-09-16T04:53:17.614302+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:17.614302+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38556882,"num_objects":9415,"num_object_clones":0,"num_object_copies":18830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10538,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2735},{"shards":"2","objects":6119}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10567","reported_seq":17424,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:17.593726+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:17.593726+0000","last_peered":"2024-09-16T04:53:17.593726+0000","last_clean":"2024-09-16T04:53:17.593726+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:17.593726+0000","last_undegraded":"2024-09-16T04:53:17.593726+0000","last_fullsized":"2024-09-16T04:53:17.593726+0000","mapping_epoch":46,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38547456,"num_objects":9411,"num_object_clones":0,"num_object_copies":18822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7290,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10516","reported_seq":10613,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:17.617310+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:17.617310+0000","last_peered":"2024-09-16T04:53:17.617310+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:17.617310+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38339177,"num_objects":9361,"num_object_clones":0,"num_object_copies":18722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10515,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2830},{"shards":"2","objects":5954}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10836","reported_seq":10939,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:17.573299+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:17.573299+0000","last_peered":"2024-09-16T04:53:17.573299+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:17.573299+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39716667,"num_objects":9699,"num_object_clones":0,"num_object_copies":19398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10833,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2948},{"shards":"2","objects":6185}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10670","reported_seq":14917,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:18.601119+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:18.601119+0000","last_peered":"2024-09-16T04:53:18.601119+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:18.601119+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39026688,"num_objects":9528,"num_object_clones":0,"num_object_copies":19056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":9528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7424,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2057,"num_bytes_recovered":8425472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8260},{"shards":"0,2","objects":1266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'10914","reported_seq":11054,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:18.583023+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:18.583023+0000","last_peered":"2024-09-16T04:53:18.583023+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:18.583023+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38920809,"num_objects":9503,"num_object_clones":0,"num_object_copies":19006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7683,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5979},{"shards":"0,2","objects":2824}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":619507959,"num_objects":151254,"num_object_clones":0,"num_object_copies":302508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":151254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":135376,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5412,"num_bytes_recovered":22134784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1285218304,"data_stored":1285137101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":772584,"kb_used_data":461840,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93599256,"statfs":{"total":96636764160,"available":95845638144,"internally_reserved":0,"allocated":472924160,"data_stored":472651105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593774,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598164,"kb_used_data":336004,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93773676,"statfs":{"total":96636764160,"available":96024244224,"internally_reserved":0,"allocated":344068096,"data_stored":343732419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182464,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":766772,"kb_used_data":460256,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93605068,"statfs":{"total":96636764160,"available":95851589632,"internally_reserved":0,"allocated":471302144,"data_stored":471033161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":469684224,"data_stored":469680745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":343646208,"data_stored":343576035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471887872,"data_stored":471880321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:22.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:53:22.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'10479", 'reported_seq': 10559, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:18.602478+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:53:18.602478+0000', 'last_peered': '2024-09-16T04:53:18.602478+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:53:18.602478+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37924864, 'num_objects': 9259, 'num_object_clones': 0, 'num_object_copies': 18518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10479, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5918}, {'shards': '0,2', 'objects': 2730}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:22.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:53:22.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'10465", 'reported_seq': 10603, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:17.617461+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:53:17.617461+0000', 'last_peered': '2024-09-16T04:53:17.617461+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:53:17.617461+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38469656, 'num_objects': 9393, 'num_object_clones': 0, 'num_object_copies': 18786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9393, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7204, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2785}, {'shards': '2', 'objects': 6074}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:22.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:53:22.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'10747", 'reported_seq': 10881, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:18.497761+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:53:18.497761+0000', 'last_peered': '2024-09-16T04:53:18.497761+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:53:18.497761+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39182336, 'num_objects': 9566, 'num_object_clones': 0, 'num_object_copies': 19132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7740, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6110}, {'shards': '0,2', 'objects': 2868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:22.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:53:22.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'10563", 'reported_seq': 10712, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:18.561820+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:53:18.561820+0000', 'last_peered': '2024-09-16T04:53:18.561820+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:53:18.561820+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38080512, 'num_objects': 9297, 'num_object_clones': 0, 'num_object_copies': 18594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9297, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7218, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5898}, {'shards': '0,2', 'objects': 2769}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:22.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:53:22.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'10638", 'reported_seq': 10866, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:17.613148+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:53:17.613148+0000', 'last_peered': '2024-09-16T04:53:17.613148+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:53:17.613148+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38678528, 'num_objects': 9443, 'num_object_clones': 0, 'num_object_copies': 18886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9443, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7389, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2740}, {'shards': '2', 'objects': 6122}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:22.816 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:53:22.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'10759", 'reported_seq': 10889, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:18.454770+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:53:18.454770+0000', 'last_peered': '2024-09-16T04:53:18.454770+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:53:18.454770+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38744064, 'num_objects': 9459, 'num_object_clones': 0, 'num_object_copies': 18918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9459, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7414, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5994}, {'shards': '0,2', 'objects': 2819}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:22.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:53:22.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'10566", 'reported_seq': 10808, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:17.614302+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:53:17.614302+0000', 'last_peered': '2024-09-16T04:53:17.614302+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:53:17.614302+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38556882, 'num_objects': 9415, 'num_object_clones': 0, 'num_object_copies': 18830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9415, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10538, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2735}, {'shards': '2', 'objects': 6119}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:22.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:53:22.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'10516", 'reported_seq': 10613, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:17.617310+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:53:17.617310+0000', 'last_peered': '2024-09-16T04:53:17.617310+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:53:17.617310+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38339177, 'num_objects': 9361, 'num_object_clones': 0, 'num_object_copies': 18722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9361, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10515, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2830}, {'shards': '2', 'objects': 5954}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:22.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:53:22.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'10836", 'reported_seq': 10939, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:17.573299+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:53:17.573299+0000', 'last_peered': '2024-09-16T04:53:17.573299+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:53:17.573299+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39716667, 'num_objects': 9699, 'num_object_clones': 0, 'num_object_copies': 19398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9699, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10833, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2948}, {'shards': '2', 'objects': 6185}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:22.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:53:22.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'10670", 'reported_seq': 14917, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:18.601119+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:53:18.601119+0000', 'last_peered': '2024-09-16T04:53:18.601119+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:53:18.601119+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39026688, 'num_objects': 9528, 'num_object_clones': 0, 'num_object_copies': 19056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1266, 'num_objects_unfound': 0, 'num_objects_dirty': 9528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7424, 'num_write_kb': 27420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2057, 'num_bytes_recovered': 8425472, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8260}, {'shards': '0,2', 'objects': 1266}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:22.817 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:53:22.818 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'10914", 'reported_seq': 11054, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:18.583023+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:53:18.583023+0000', 'last_peered': '2024-09-16T04:53:18.583023+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:53:18.583023+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38920809, 'num_objects': 9503, 'num_object_clones': 0, 'num_object_copies': 19006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9503, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7683, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5979}, {'shards': '0,2', 'objects': 2824}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:22.818 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:23.617 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/302512 objects degraded (11.260%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":351,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":151256,"data_bytes":619507978,"bytes_used":2188820480,"bytes_avail":287721472000,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":302512,"degraded_ratio":0.11259718622732322,"misplaced_objects":1266,"misplaced_total":302512,"misplaced_ratio":0.0041849579520812398,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":83946,"recovering_keys_per_sec":0,"num_objects_recovered":246,"num_bytes_recovered":1007616,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":375},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:53:23.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:53:23.670 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:24.552 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:24.554 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:24.651 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-16T04:53:24.373661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":610218250,"num_objects":148988,"num_object_clones":0,"num_object_copies":297976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1150,"num_objects_unfound":0,"num_objects_dirty":148988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137646,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5531,"num_bytes_recovered":22609939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160969,"ondisk_log_size":160969,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2121856,"kb_used_data":1246596,"kb_used_omap":12,"kb_used_meta":875187,"kb_avail":280993664,"statfs":{"total":289910292480,"available":287737511936,"internally_reserved":0,"allocated":1276514304,"data_stored":1275636589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":896191489},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,11,12,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27762688,"num_objects":-6778,"num_object_clones":0,"num_object_copies":-13556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-362,"num_objects_unfound":0,"num_objects_dirty":-6778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6778,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":364,"num_bytes_recovered":1482752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002986"},"pg_stats":[{"pgid":"2.f","version":"47'10627","reported_seq":10707,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.633430+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:23.633430+0000","last_peered":"2024-09-16T04:53:23.633430+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:23.633430+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37318656,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10627,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5770},{"shards":"0,2","objects":2582}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10720","reported_seq":10837,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:22.657604+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:22.657604+0000","last_peered":"2024-09-16T04:53:22.657604+0000","last_clean":"2024-09-16T04:53:22.657604+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:22.657604+0000","last_undegraded":"2024-09-16T04:53:22.657604+0000","last_fullsized":"2024-09-16T04:53:22.657604+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37986304,"num_objects":9274,"num_object_clones":0,"num_object_copies":18548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7689,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10615","reported_seq":10755,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:22.678461+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:22.678461+0000","last_peered":"2024-09-16T04:53:22.678461+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:22.678461+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37855256,"num_objects":9243,"num_object_clones":0,"num_object_copies":18486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7354,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2635},{"shards":"2","objects":5924}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'10875","reported_seq":10946,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:22.696898+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:22.696898+0000","last_peered":"2024-09-16T04:53:22.696898+0000","last_clean":"2024-09-16T04:53:22.696898+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:22.696898+0000","last_undegraded":"2024-09-16T04:53:22.696898+0000","last_fullsized":"2024-09-16T04:53:22.696898+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38785024,"num_objects":9469,"num_object_clones":0,"num_object_copies":18938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10875,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10811","reported_seq":11030,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:23.415400+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:23.415400+0000","last_peered":"2024-09-16T04:53:23.415400+0000","last_clean":"2024-09-16T04:53:23.415400+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:23.415400+0000","last_undegraded":"2024-09-16T04:53:23.415400+0000","last_fullsized":"2024-09-16T04:53:23.415400+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38227968,"num_objects":9333,"num_object_clones":0,"num_object_copies":18666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7843,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10895","reported_seq":11027,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.605461+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:23.605461+0000","last_peered":"2024-09-16T04:53:23.605461+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:23.605461+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38576128,"num_objects":9418,"num_object_clones":0,"num_object_copies":18836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7888,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5962},{"shards":"0,2","objects":2720}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10700","reported_seq":10849,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.600718+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:23.600718+0000","last_peered":"2024-09-16T04:53:23.600718+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:23.600718+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37519360,"num_objects":9160,"num_object_clones":0,"num_object_copies":18320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7355,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5761},{"shards":"0,2","objects":2632}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10748","reported_seq":10964,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:23.580799+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:23.580799+0000","last_peered":"2024-09-16T04:53:23.580799+0000","last_clean":"2024-09-16T04:53:23.580799+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:23.580799+0000","last_undegraded":"2024-09-16T04:53:23.580799+0000","last_fullsized":"2024-09-16T04:53:23.580799+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37961728,"num_objects":9268,"num_object_clones":0,"num_object_copies":18536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7818,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10805","reported_seq":11033,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:22.673849+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:22.673849+0000","last_peered":"2024-09-16T04:53:22.673849+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:22.673849+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37994496,"num_objects":9276,"num_object_clones":0,"num_object_copies":18552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7556,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2573},{"shards":"2","objects":5955}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10891","reported_seq":11020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.612617+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:23.612617+0000","last_peered":"2024-09-16T04:53:23.612617+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:23.612617+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38203392,"num_objects":9327,"num_object_clones":0,"num_object_copies":18654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7546,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5862},{"shards":"0,2","objects":2687}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10697","reported_seq":10939,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:22.573084+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:22.573084+0000","last_peered":"2024-09-16T04:53:22.573084+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:22.573084+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38020306,"num_objects":9284,"num_object_clones":0,"num_object_copies":18568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10669,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2604},{"shards":"2","objects":5988}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10698","reported_seq":17555,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:22.690251+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:22.690251+0000","last_peered":"2024-09-16T04:53:22.690251+0000","last_clean":"2024-09-16T04:53:22.690251+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:22.690251+0000","last_undegraded":"2024-09-16T04:53:22.690251+0000","last_fullsized":"2024-09-16T04:53:22.690251+0000","mapping_epoch":46,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38010880,"num_objects":9280,"num_object_clones":0,"num_object_copies":18560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7421,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10640","reported_seq":10738,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:22.631625+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:22.631625+0000","last_peered":"2024-09-16T04:53:22.631625+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:22.631625+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37831273,"num_objects":9237,"num_object_clones":0,"num_object_copies":18474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10639,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2706},{"shards":"2","objects":5830}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'10967","reported_seq":11070,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:22.727073+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:22.727073+0000","last_peered":"2024-09-16T04:53:22.727073+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:22.727073+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39180091,"num_objects":9568,"num_object_clones":0,"num_object_copies":19136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10964,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2817},{"shards":"2","objects":6054}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10797","reported_seq":15280,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.628501+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:23.628501+0000","last_peered":"2024-09-16T04:53:23.628501+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:23.628501+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38506496,"num_objects":9401,"num_object_clones":0,"num_object_copies":18802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1150,"num_objects_unfound":0,"num_objects_dirty":9401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7551,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2175,"num_bytes_recovered":8900608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8249},{"shards":"0,2","objects":1150}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11080","reported_seq":11220,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.442104+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:23.442104+0000","last_peered":"2024-09-16T04:53:23.442104+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:23.442104+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38240873,"num_objects":9337,"num_object_clones":0,"num_object_copies":18674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7849,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5813},{"shards":"0,2","objects":2658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":610218231,"num_objects":148986,"num_object_clones":0,"num_object_copies":297972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1150,"num_objects_unfound":0,"num_objects_dirty":148986,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137644,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5530,"num_bytes_recovered":22609920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1273434112,"data_stored":1273352909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160966,"ondisk_log_size":160966,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754480,"kb_used_data":456224,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93617360,"statfs":{"total":96636764160,"available":95864176640,"internally_reserved":0,"allocated":467173376,"data_stored":466900321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593774,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598164,"kb_used_data":336004,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93773676,"statfs":{"total":96636764160,"available":96024244224,"internally_reserved":0,"allocated":344068096,"data_stored":343732419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769212,"kb_used_data":454368,"kb_used_omap":4,"kb_used_meta":314811,"kb_avail":93602628,"statfs":{"total":96636764160,"available":95849091072,"internally_reserved":0,"allocated":465272832,"data_stored":465003849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":322367146},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":463835136,"data_stored":463831657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":343646208,"data_stored":343576035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":465952768,"data_stored":465945217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:27.653 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:28.455 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:28.457 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:28.552 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-16T04:53:28.374623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605557002,"num_objects":147850,"num_object_clones":0,"num_object_copies":295700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1150,"num_objects_unfound":0,"num_objects_dirty":147850,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138784,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5531,"num_bytes_recovered":22609939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160907,"ondisk_log_size":160907,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2124016,"kb_used_data":1240436,"kb_used_omap":12,"kb_used_meta":883507,"kb_avail":280991504,"statfs":{"total":289910292480,"available":287735300096,"internally_reserved":0,"allocated":1270206464,"data_stored":1269328749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":904711169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,12,8,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23080960,"num_objects":-5635,"num_object_clones":0,"num_object_copies":-11270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-237,"num_objects_unfound":0,"num_objects_dirty":-5635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5635,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":970752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002925"},"pg_stats":[{"pgid":"2.f","version":"47'10627","reported_seq":10707,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.633430+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:23.633430+0000","last_peered":"2024-09-16T04:53:23.633430+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:23.633430+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37318656,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10627,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5770},{"shards":"0,2","objects":2582}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10850","reported_seq":10967,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:27.745766+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:27.745766+0000","last_peered":"2024-09-16T04:53:27.745766+0000","last_clean":"2024-09-16T04:53:27.745766+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:27.745766+0000","last_undegraded":"2024-09-16T04:53:27.745766+0000","last_fullsized":"2024-09-16T04:53:27.745766+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7819,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10760","reported_seq":10900,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.634449+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:27.634449+0000","last_peered":"2024-09-16T04:53:27.634449+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:27.634449+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37261336,"num_objects":9098,"num_object_clones":0,"num_object_copies":18196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9098,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7499,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2490},{"shards":"2","objects":5779}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11018","reported_seq":11089,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:27.721264+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:27.721264+0000","last_peered":"2024-09-16T04:53:27.721264+0000","last_clean":"2024-09-16T04:53:27.721264+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:27.721264+0000","last_undegraded":"2024-09-16T04:53:27.721264+0000","last_fullsized":"2024-09-16T04:53:27.721264+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38199296,"num_objects":9326,"num_object_clones":0,"num_object_copies":18652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11018,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10811","reported_seq":11030,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:23.415400+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:23.415400+0000","last_peered":"2024-09-16T04:53:23.415400+0000","last_clean":"2024-09-16T04:53:23.415400+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:23.415400+0000","last_undegraded":"2024-09-16T04:53:23.415400+0000","last_fullsized":"2024-09-16T04:53:23.415400+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38227968,"num_objects":9333,"num_object_clones":0,"num_object_copies":18666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7843,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10895","reported_seq":11027,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.605461+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:23.605461+0000","last_peered":"2024-09-16T04:53:23.605461+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:23.605461+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38576128,"num_objects":9418,"num_object_clones":0,"num_object_copies":18836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7888,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5962},{"shards":"0,2","objects":2720}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10700","reported_seq":10849,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.600718+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:23.600718+0000","last_peered":"2024-09-16T04:53:23.600718+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:23.600718+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37519360,"num_objects":9160,"num_object_clones":0,"num_object_copies":18320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7355,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5761},{"shards":"0,2","objects":2632}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10748","reported_seq":10964,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:23.580799+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:23.580799+0000","last_peered":"2024-09-16T04:53:23.580799+0000","last_clean":"2024-09-16T04:53:23.580799+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:23.580799+0000","last_undegraded":"2024-09-16T04:53:23.580799+0000","last_fullsized":"2024-09-16T04:53:23.580799+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37961728,"num_objects":9268,"num_object_clones":0,"num_object_copies":18536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7818,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10940","reported_seq":11168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.677649+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:27.677649+0000","last_peered":"2024-09-16T04:53:27.677649+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:27.677649+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37441536,"num_objects":9141,"num_object_clones":0,"num_object_copies":18282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7691,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2438},{"shards":"2","objects":5820}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10891","reported_seq":11020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.612617+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:23.612617+0000","last_peered":"2024-09-16T04:53:23.612617+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:23.612617+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38203392,"num_objects":9327,"num_object_clones":0,"num_object_copies":18654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7546,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5862},{"shards":"0,2","objects":2687}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10829","reported_seq":11071,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.633128+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:27.633128+0000","last_peered":"2024-09-16T04:53:27.633128+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:27.633128+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37479634,"num_objects":9152,"num_object_clones":0,"num_object_copies":18304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10801,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2472},{"shards":"2","objects":5856}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10849","reported_seq":17706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:27.583747+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:27.583747+0000","last_peered":"2024-09-16T04:53:27.583747+0000","last_clean":"2024-09-16T04:53:27.583747+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:27.583747+0000","last_undegraded":"2024-09-16T04:53:27.583747+0000","last_fullsized":"2024-09-16T04:53:27.583747+0000","mapping_epoch":46,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37392384,"num_objects":9129,"num_object_clones":0,"num_object_copies":18258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7572,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10791","reported_seq":10889,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.684028+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:27.684028+0000","last_peered":"2024-09-16T04:53:27.684028+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:27.684028+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37212777,"num_objects":9086,"num_object_clones":0,"num_object_copies":18172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10790,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2555},{"shards":"2","objects":5679}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11118","reported_seq":11221,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.696016+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:27.696016+0000","last_peered":"2024-09-16T04:53:27.696016+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:27.696016+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38561595,"num_objects":9417,"num_object_clones":0,"num_object_copies":18834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11115,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2666},{"shards":"2","objects":5903}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10797","reported_seq":15280,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.628501+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:23.628501+0000","last_peered":"2024-09-16T04:53:23.628501+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:23.628501+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38506496,"num_objects":9401,"num_object_clones":0,"num_object_copies":18802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1150,"num_objects_unfound":0,"num_objects_dirty":9401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7551,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2175,"num_bytes_recovered":8900608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8249},{"shards":"0,2","objects":1150}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11080","reported_seq":11220,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.442104+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:23.442104+0000","last_peered":"2024-09-16T04:53:23.442104+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:23.442104+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38240873,"num_objects":9337,"num_object_clones":0,"num_object_copies":18674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7849,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5813},{"shards":"0,2","objects":2658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":605556983,"num_objects":147848,"num_object_clones":0,"num_object_copies":295696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1150,"num_objects_unfound":0,"num_objects_dirty":147848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138782,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5530,"num_bytes_recovered":22609920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1267081216,"data_stored":1267000013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":753112,"kb_used_data":450696,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93618728,"statfs":{"total":96636764160,"available":95865577472,"internally_reserved":0,"allocated":461512704,"data_stored":461239649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593775,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601692,"kb_used_data":335372,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93770148,"statfs":{"total":96636764160,"available":96020631552,"internally_reserved":0,"allocated":343420928,"data_stored":343085251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769212,"kb_used_data":454368,"kb_used_omap":4,"kb_used_meta":314811,"kb_avail":93602628,"statfs":{"total":96636764160,"available":95849091072,"internally_reserved":0,"allocated":465272832,"data_stored":465003849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":322367146},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":463835136,"data_stored":463831657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":342982656,"data_stored":342912483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":460263424,"data_stored":460255873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:28.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:29.598 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:29.599 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:29.666 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-16T04:53:28.374623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605557002,"num_objects":147850,"num_object_clones":0,"num_object_copies":295700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1150,"num_objects_unfound":0,"num_objects_dirty":147850,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138784,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5531,"num_bytes_recovered":22609939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160907,"ondisk_log_size":160907,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2124016,"kb_used_data":1240436,"kb_used_omap":12,"kb_used_meta":883507,"kb_avail":280991504,"statfs":{"total":289910292480,"available":287735300096,"internally_reserved":0,"allocated":1270206464,"data_stored":1269328749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":904711169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,12,8,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23080960,"num_objects":-5635,"num_object_clones":0,"num_object_copies":-11270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-237,"num_objects_unfound":0,"num_objects_dirty":-5635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5635,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":970752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002925"},"pg_stats":[{"pgid":"2.f","version":"47'10627","reported_seq":10707,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.633430+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:23.633430+0000","last_peered":"2024-09-16T04:53:23.633430+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:23.633430+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37318656,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10627,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5770},{"shards":"0,2","objects":2582}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10850","reported_seq":10967,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:27.745766+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:27.745766+0000","last_peered":"2024-09-16T04:53:27.745766+0000","last_clean":"2024-09-16T04:53:27.745766+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:27.745766+0000","last_undegraded":"2024-09-16T04:53:27.745766+0000","last_fullsized":"2024-09-16T04:53:27.745766+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7819,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10760","reported_seq":10900,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.634449+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:27.634449+0000","last_peered":"2024-09-16T04:53:27.634449+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:27.634449+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37261336,"num_objects":9098,"num_object_clones":0,"num_object_copies":18196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9098,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7499,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2490},{"shards":"2","objects":5779}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11018","reported_seq":11089,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:27.721264+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:27.721264+0000","last_peered":"2024-09-16T04:53:27.721264+0000","last_clean":"2024-09-16T04:53:27.721264+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:27.721264+0000","last_undegraded":"2024-09-16T04:53:27.721264+0000","last_fullsized":"2024-09-16T04:53:27.721264+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38199296,"num_objects":9326,"num_object_clones":0,"num_object_copies":18652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11018,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10811","reported_seq":11030,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:23.415400+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:23.415400+0000","last_peered":"2024-09-16T04:53:23.415400+0000","last_clean":"2024-09-16T04:53:23.415400+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:23.415400+0000","last_undegraded":"2024-09-16T04:53:23.415400+0000","last_fullsized":"2024-09-16T04:53:23.415400+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38227968,"num_objects":9333,"num_object_clones":0,"num_object_copies":18666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7843,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'10895","reported_seq":11027,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.605461+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:23.605461+0000","last_peered":"2024-09-16T04:53:23.605461+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:23.605461+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38576128,"num_objects":9418,"num_object_clones":0,"num_object_copies":18836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7888,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5962},{"shards":"0,2","objects":2720}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10700","reported_seq":10849,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.600718+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:23.600718+0000","last_peered":"2024-09-16T04:53:23.600718+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:23.600718+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37519360,"num_objects":9160,"num_object_clones":0,"num_object_copies":18320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7355,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5761},{"shards":"0,2","objects":2632}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10748","reported_seq":10964,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:23.580799+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:23.580799+0000","last_peered":"2024-09-16T04:53:23.580799+0000","last_clean":"2024-09-16T04:53:23.580799+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:23.580799+0000","last_undegraded":"2024-09-16T04:53:23.580799+0000","last_fullsized":"2024-09-16T04:53:23.580799+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37961728,"num_objects":9268,"num_object_clones":0,"num_object_copies":18536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7818,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10940","reported_seq":11168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.677649+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:27.677649+0000","last_peered":"2024-09-16T04:53:27.677649+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:27.677649+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37441536,"num_objects":9141,"num_object_clones":0,"num_object_copies":18282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7691,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2438},{"shards":"2","objects":5820}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'10891","reported_seq":11020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.612617+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:23.612617+0000","last_peered":"2024-09-16T04:53:23.612617+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:23.612617+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38203392,"num_objects":9327,"num_object_clones":0,"num_object_copies":18654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7546,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5862},{"shards":"0,2","objects":2687}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10829","reported_seq":11071,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.633128+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:27.633128+0000","last_peered":"2024-09-16T04:53:27.633128+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:27.633128+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37479634,"num_objects":9152,"num_object_clones":0,"num_object_copies":18304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10801,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2472},{"shards":"2","objects":5856}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10849","reported_seq":17706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:27.583747+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:27.583747+0000","last_peered":"2024-09-16T04:53:27.583747+0000","last_clean":"2024-09-16T04:53:27.583747+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:27.583747+0000","last_undegraded":"2024-09-16T04:53:27.583747+0000","last_fullsized":"2024-09-16T04:53:27.583747+0000","mapping_epoch":46,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37392384,"num_objects":9129,"num_object_clones":0,"num_object_copies":18258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7572,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10791","reported_seq":10889,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.684028+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:27.684028+0000","last_peered":"2024-09-16T04:53:27.684028+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:27.684028+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37212777,"num_objects":9086,"num_object_clones":0,"num_object_copies":18172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10790,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2555},{"shards":"2","objects":5679}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11118","reported_seq":11221,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.696016+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:27.696016+0000","last_peered":"2024-09-16T04:53:27.696016+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:27.696016+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38561595,"num_objects":9417,"num_object_clones":0,"num_object_copies":18834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11115,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2666},{"shards":"2","objects":5903}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10797","reported_seq":15280,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.628501+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:23.628501+0000","last_peered":"2024-09-16T04:53:23.628501+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:23.628501+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38506496,"num_objects":9401,"num_object_clones":0,"num_object_copies":18802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1150,"num_objects_unfound":0,"num_objects_dirty":9401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7551,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2175,"num_bytes_recovered":8900608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8249},{"shards":"0,2","objects":1150}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11080","reported_seq":11220,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:23.442104+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:23.442104+0000","last_peered":"2024-09-16T04:53:23.442104+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:23.442104+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38240873,"num_objects":9337,"num_object_clones":0,"num_object_copies":18674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7849,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5813},{"shards":"0,2","objects":2658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":605556983,"num_objects":147848,"num_object_clones":0,"num_object_copies":295696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1150,"num_objects_unfound":0,"num_objects_dirty":147848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138782,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5530,"num_bytes_recovered":22609920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1267081216,"data_stored":1267000013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":753112,"kb_used_data":450696,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93618728,"statfs":{"total":96636764160,"available":95865577472,"internally_reserved":0,"allocated":461512704,"data_stored":461239649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593775,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601692,"kb_used_data":335372,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93770148,"statfs":{"total":96636764160,"available":96020631552,"internally_reserved":0,"allocated":343420928,"data_stored":343085251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":769212,"kb_used_data":454368,"kb_used_omap":4,"kb_used_meta":314811,"kb_avail":93602628,"statfs":{"total":96636764160,"available":95849091072,"internally_reserved":0,"allocated":465272832,"data_stored":465003849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":322367146},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":463835136,"data_stored":463831657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":342982656,"data_stored":342912483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":460263424,"data_stored":460255873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:29.668 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:53:29.668 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'10627", 'reported_seq': 10707, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:23.633430+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:53:23.633430+0000', 'last_peered': '2024-09-16T04:53:23.633430+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:53:23.633430+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37318656, 'num_objects': 9111, 'num_object_clones': 0, 'num_object_copies': 18222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9111, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10627, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5770}, {'shards': '0,2', 'objects': 2582}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:29.668 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:53:29.668 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'10760", 'reported_seq': 10900, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:27.634449+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:53:27.634449+0000', 'last_peered': '2024-09-16T04:53:27.634449+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:53:27.634449+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37261336, 'num_objects': 9098, 'num_object_clones': 0, 'num_object_copies': 18196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9098, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7499, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2490}, {'shards': '2', 'objects': 5779}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:29.668 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:53:29.668 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'10895", 'reported_seq': 11027, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:23.605461+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:53:23.605461+0000', 'last_peered': '2024-09-16T04:53:23.605461+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:53:23.605461+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38576128, 'num_objects': 9418, 'num_object_clones': 0, 'num_object_copies': 18836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9418, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7888, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5962}, {'shards': '0,2', 'objects': 2720}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:29.668 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:53:29.668 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'10700", 'reported_seq': 10849, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:23.600718+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:53:23.600718+0000', 'last_peered': '2024-09-16T04:53:23.600718+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:53:23.600718+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37519360, 'num_objects': 9160, 'num_object_clones': 0, 'num_object_copies': 18320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7355, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5761}, {'shards': '0,2', 'objects': 2632}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:29.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:53:29.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'10940", 'reported_seq': 11168, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:27.677649+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:53:27.677649+0000', 'last_peered': '2024-09-16T04:53:27.677649+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:53:27.677649+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37441536, 'num_objects': 9141, 'num_object_clones': 0, 'num_object_copies': 18282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9141, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7691, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2438}, {'shards': '2', 'objects': 5820}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:29.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:53:29.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'10891", 'reported_seq': 11020, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:23.612617+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:53:23.612617+0000', 'last_peered': '2024-09-16T04:53:23.612617+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:53:23.612617+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38203392, 'num_objects': 9327, 'num_object_clones': 0, 'num_object_copies': 18654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9327, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7546, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5862}, {'shards': '0,2', 'objects': 2687}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:29.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:53:29.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'10829", 'reported_seq': 11071, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:27.633128+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:53:27.633128+0000', 'last_peered': '2024-09-16T04:53:27.633128+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:53:27.633128+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37479634, 'num_objects': 9152, 'num_object_clones': 0, 'num_object_copies': 18304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9152, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10801, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2472}, {'shards': '2', 'objects': 5856}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:29.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:53:29.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'10791", 'reported_seq': 10889, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:27.684028+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:53:27.684028+0000', 'last_peered': '2024-09-16T04:53:27.684028+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:53:27.684028+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37212777, 'num_objects': 9086, 'num_object_clones': 0, 'num_object_copies': 18172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9086, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10790, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2555}, {'shards': '2', 'objects': 5679}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:29.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:53:29.669 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'11118", 'reported_seq': 11221, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:27.696016+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:53:27.696016+0000', 'last_peered': '2024-09-16T04:53:27.696016+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:53:27.696016+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38561595, 'num_objects': 9417, 'num_object_clones': 0, 'num_object_copies': 18834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9417, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11115, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2666}, {'shards': '2', 'objects': 5903}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:29.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:53:29.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'10797", 'reported_seq': 15280, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:23.628501+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:53:23.628501+0000', 'last_peered': '2024-09-16T04:53:23.628501+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:53:23.628501+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38506496, 'num_objects': 9401, 'num_object_clones': 0, 'num_object_copies': 18802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 1150, 'num_objects_unfound': 0, 'num_objects_dirty': 9401, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7551, 'num_write_kb': 27420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2175, 'num_bytes_recovered': 8900608, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8249}, {'shards': '0,2', 'objects': 1150}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:29.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:53:29.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'11080", 'reported_seq': 11220, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:23.442104+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:53:23.442104+0000', 'last_peered': '2024-09-16T04:53:23.442104+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:53:23.442104+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38240873, 'num_objects': 9337, 'num_object_clones': 0, 'num_object_copies': 18674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9337, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7849, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5813}, {'shards': '0,2', 'objects': 2658}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:29.670 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:53:30.730 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:30.730 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/293402 objects degraded (11.609%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":358,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":146701,"data_bytes":600850698,"bytes_used":2155405312,"bytes_avail":287754887168,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":293402,"degraded_ratio":0.11609327816442969,"misplaced_objects":1015,"misplaced_total":293402,"misplaced_ratio":0.0034594174545504119,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":126923,"recovering_keys_per_sec":0,"num_objects_recovered":374,"num_bytes_recovered":1523712,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":476},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:53:30.763 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:53:30.763 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:31.577 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:31.579 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:31.629 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-16T04:53:30.377153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600850698,"num_objects":146701,"num_object_clones":0,"num_object_copies":293402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1015,"num_objects_unfound":0,"num_objects_dirty":146701,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":139933,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5666,"num_bytes_recovered":23162899,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160756,"ondisk_log_size":160756,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2104888,"kb_used_data":1233792,"kb_used_omap":12,"kb_used_meta":871027,"kb_avail":281010632,"statfs":{"total":289910292480,"available":287754887168,"internally_reserved":0,"allocated":1263403008,"data_stored":1262525293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":891931649},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,3,15,10,10,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23441408,"num_objects":-5723,"num_object_clones":0,"num_object_copies":-11446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-372,"num_objects_unfound":0,"num_objects_dirty":-5723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5723,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":1523712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004947"},"pg_stats":[{"pgid":"2.f","version":"47'10768","reported_seq":10848,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:28.604482+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:28.604482+0000","last_peered":"2024-09-16T04:53:28.604482+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:28.604482+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36741120,"num_objects":8970,"num_object_clones":0,"num_object_copies":17940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10768,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5629},{"shards":"0,2","objects":2441}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'10850","reported_seq":10967,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:27.745766+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:27.745766+0000","last_peered":"2024-09-16T04:53:27.745766+0000","last_clean":"2024-09-16T04:53:27.745766+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:27.745766+0000","last_undegraded":"2024-09-16T04:53:27.745766+0000","last_fullsized":"2024-09-16T04:53:27.745766+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37453824,"num_objects":9144,"num_object_clones":0,"num_object_copies":18288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7819,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10760","reported_seq":10900,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.634449+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:27.634449+0000","last_peered":"2024-09-16T04:53:27.634449+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:27.634449+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37261336,"num_objects":9098,"num_object_clones":0,"num_object_copies":18196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9098,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7499,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2490},{"shards":"2","objects":5779}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11018","reported_seq":11089,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:27.721264+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:27.721264+0000","last_peered":"2024-09-16T04:53:27.721264+0000","last_clean":"2024-09-16T04:53:27.721264+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:27.721264+0000","last_undegraded":"2024-09-16T04:53:27.721264+0000","last_fullsized":"2024-09-16T04:53:27.721264+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38199296,"num_objects":9326,"num_object_clones":0,"num_object_copies":18652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11018,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'10948","reported_seq":11167,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:28.496590+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:28.496590+0000","last_peered":"2024-09-16T04:53:28.496590+0000","last_clean":"2024-09-16T04:53:28.496590+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:28.496590+0000","last_undegraded":"2024-09-16T04:53:28.496590+0000","last_fullsized":"2024-09-16T04:53:28.496590+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37666816,"num_objects":9196,"num_object_clones":0,"num_object_copies":18392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7980,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11057","reported_seq":11191,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:28.561496+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:28.561496+0000","last_peered":"2024-09-16T04:53:28.561496+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:28.561496+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37912576,"num_objects":9256,"num_object_clones":0,"num_object_copies":18512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8050,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5800},{"shards":"0,2","objects":2558}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10846","reported_seq":10995,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:28.598724+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:28.598724+0000","last_peered":"2024-09-16T04:53:28.598724+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:28.598724+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36921344,"num_objects":9014,"num_object_clones":0,"num_object_copies":18028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7501,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5615},{"shards":"0,2","objects":2486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'10867","reported_seq":11083,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:28.532845+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:28.532845+0000","last_peered":"2024-09-16T04:53:28.532845+0000","last_clean":"2024-09-16T04:53:28.532845+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:28.532845+0000","last_undegraded":"2024-09-16T04:53:28.532845+0000","last_fullsized":"2024-09-16T04:53:28.532845+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37474304,"num_objects":9149,"num_object_clones":0,"num_object_copies":18298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7937,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'10940","reported_seq":11168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.677649+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:27.677649+0000","last_peered":"2024-09-16T04:53:27.677649+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:27.677649+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37441536,"num_objects":9141,"num_object_clones":0,"num_object_copies":18282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7691,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2438},{"shards":"2","objects":5820}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11043","reported_seq":11173,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:28.577664+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:28.577664+0000","last_peered":"2024-09-16T04:53:28.577664+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:28.577664+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37580800,"num_objects":9175,"num_object_clones":0,"num_object_copies":18350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7698,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5710},{"shards":"0,2","objects":2535}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10829","reported_seq":11071,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.633128+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:27.633128+0000","last_peered":"2024-09-16T04:53:27.633128+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:27.633128+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37479634,"num_objects":9152,"num_object_clones":0,"num_object_copies":18304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10801,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2472},{"shards":"2","objects":5856}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10849","reported_seq":17706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:27.583747+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:27.583747+0000","last_peered":"2024-09-16T04:53:27.583747+0000","last_clean":"2024-09-16T04:53:27.583747+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:27.583747+0000","last_undegraded":"2024-09-16T04:53:27.583747+0000","last_fullsized":"2024-09-16T04:53:27.583747+0000","mapping_epoch":46,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37392384,"num_objects":9129,"num_object_clones":0,"num_object_copies":18258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7572,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10791","reported_seq":10889,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.684028+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:27.684028+0000","last_peered":"2024-09-16T04:53:27.684028+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:27.684028+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37212777,"num_objects":9086,"num_object_clones":0,"num_object_copies":18172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10790,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2555},{"shards":"2","objects":5679}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11118","reported_seq":11221,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:27.696016+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:27.696016+0000","last_peered":"2024-09-16T04:53:27.696016+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:27.696016+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38561595,"num_objects":9417,"num_object_clones":0,"num_object_copies":18834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11115,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2666},{"shards":"2","objects":5903}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'10936","reported_seq":15688,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:28.608724+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:28.608724+0000","last_peered":"2024-09-16T04:53:28.608724+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:28.608724+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37937152,"num_objects":9262,"num_object_clones":0,"num_object_copies":18524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":1015,"num_objects_unfound":0,"num_objects_dirty":9262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7690,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2310,"num_bytes_recovered":9453568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8245},{"shards":"0,2","objects":1015}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11233","reported_seq":11373,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:28.535166+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:28.535166+0000","last_peered":"2024-09-16T04:53:28.535166+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:28.535166+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37614185,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8002,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5660},{"shards":"0,2","objects":2505}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":600850679,"num_objects":146699,"num_object_clones":0,"num_object_copies":293398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":1015,"num_objects_unfound":0,"num_objects_dirty":146699,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":139931,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5665,"num_bytes_recovered":23162880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1261215744,"data_stored":1261134541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":753112,"kb_used_data":450696,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93618728,"statfs":{"total":96636764160,"available":95865577472,"internally_reserved":0,"allocated":461512704,"data_stored":461239649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593775,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601692,"kb_used_data":335372,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93770148,"statfs":{"total":96636764160,"available":96020631552,"internally_reserved":0,"allocated":343420928,"data_stored":343085251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":750084,"kb_used_data":447724,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93621756,"statfs":{"total":96636764160,"available":95868678144,"internally_reserved":0,"allocated":458469376,"data_stored":458200393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,5,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":457969664,"data_stored":457966185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":342982656,"data_stored":342912483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":460263424,"data_stored":460255873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:34.632 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:35.508 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:35.514 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:35.559 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-16T04:53:34.378109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591585546,"num_objects":144439,"num_object_clones":0,"num_object_copies":288878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":877,"num_objects_unfound":0,"num_objects_dirty":144439,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142195,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5804,"num_bytes_recovered":23728147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110736,"kb_used_data":1222992,"kb_used_omap":12,"kb_used_meta":887667,"kb_avail":281004784,"statfs":{"total":289910292480,"available":287748898816,"internally_reserved":0,"allocated":1252343808,"data_stored":1251466093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":908971009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,4,4,2,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27922432,"num_objects":-6817,"num_object_clones":0,"num_object_copies":-13634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-389,"num_objects_unfound":0,"num_objects_dirty":-6817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6817,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":1593344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004928"},"pg_stats":[{"pgid":"2.f","version":"47'10897","reported_seq":10976,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.572045+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:33.572045+0000","last_peered":"2024-09-16T04:53:33.572045+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:33.572045+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36212736,"num_objects":8841,"num_object_clones":0,"num_object_copies":17682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10897,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5500},{"shards":"0,2","objects":2312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11003","reported_seq":11120,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:32.754840+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:32.754840+0000","last_peered":"2024-09-16T04:53:32.754840+0000","last_clean":"2024-09-16T04:53:32.754840+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:32.754840+0000","last_undegraded":"2024-09-16T04:53:32.754840+0000","last_fullsized":"2024-09-16T04:53:32.754840+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36827136,"num_objects":8991,"num_object_clones":0,"num_object_copies":17982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7972,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10899","reported_seq":11039,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.580284+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:32.580284+0000","last_peered":"2024-09-16T04:53:32.580284+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:32.580284+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36691992,"num_objects":8959,"num_object_clones":0,"num_object_copies":17918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7638,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2351},{"shards":"2","objects":5640}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11170","reported_seq":11241,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:32.743336+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:32.743336+0000","last_peered":"2024-09-16T04:53:32.743336+0000","last_clean":"2024-09-16T04:53:32.743336+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:32.743336+0000","last_undegraded":"2024-09-16T04:53:32.743336+0000","last_fullsized":"2024-09-16T04:53:32.743336+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37576704,"num_objects":9174,"num_object_clones":0,"num_object_copies":18348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11170,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11103","reported_seq":11322,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:33.667259+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:33.667259+0000","last_peered":"2024-09-16T04:53:33.667259+0000","last_clean":"2024-09-16T04:53:33.667259+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:33.667259+0000","last_undegraded":"2024-09-16T04:53:33.667259+0000","last_fullsized":"2024-09-16T04:53:33.667259+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37031936,"num_objects":9041,"num_object_clones":0,"num_object_copies":18082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8135,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11211","reported_seq":11345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.563016+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:33.563016+0000","last_peered":"2024-09-16T04:53:33.563016+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:33.563016+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37281792,"num_objects":9102,"num_object_clones":0,"num_object_copies":18204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8204,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5646},{"shards":"0,2","objects":2404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10973","reported_seq":11122,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.601236+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:33.601236+0000","last_peered":"2024-09-16T04:53:33.601236+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:33.601236+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36401152,"num_objects":8887,"num_object_clones":0,"num_object_copies":17774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7628,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5488},{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11015","reported_seq":11231,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:33.651407+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:33.651407+0000","last_peered":"2024-09-16T04:53:33.651407+0000","last_clean":"2024-09-16T04:53:33.651407+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:33.651407+0000","last_undegraded":"2024-09-16T04:53:33.651407+0000","last_fullsized":"2024-09-16T04:53:33.651407+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36868096,"num_objects":9001,"num_object_clones":0,"num_object_copies":18002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8085,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11065","reported_seq":11292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.714659+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:32.714659+0000","last_peered":"2024-09-16T04:53:32.714659+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:32.714659+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36929536,"num_objects":9016,"num_object_clones":0,"num_object_copies":18032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7816,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2313},{"shards":"2","objects":5695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11194","reported_seq":11324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.532774+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:33.532774+0000","last_peered":"2024-09-16T04:53:33.532774+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:33.532774+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36962304,"num_objects":9024,"num_object_clones":0,"num_object_copies":18048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7849,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5559},{"shards":"0,2","objects":2384}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10965","reported_seq":11206,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.754550+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:32.754550+0000","last_peered":"2024-09-16T04:53:32.754550+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:32.754550+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36922578,"num_objects":9016,"num_object_clones":0,"num_object_copies":18032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10937,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2336},{"shards":"2","objects":5720}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10971","reported_seq":17828,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:32.665877+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:32.665877+0000","last_peered":"2024-09-16T04:53:32.665877+0000","last_clean":"2024-09-16T04:53:32.665877+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:32.665877+0000","last_undegraded":"2024-09-16T04:53:32.665877+0000","last_fullsized":"2024-09-16T04:53:32.665877+0000","mapping_epoch":46,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36892672,"num_objects":9007,"num_object_clones":0,"num_object_copies":18014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7694,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10936","reported_seq":11034,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.753631+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:32.753631+0000","last_peered":"2024-09-16T04:53:32.753631+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:32.753631+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36618857,"num_objects":8941,"num_object_clones":0,"num_object_copies":17882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10935,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2410},{"shards":"2","objects":5534}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11269","reported_seq":11370,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.753777+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:32.753777+0000","last_peered":"2024-09-16T04:53:32.753777+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:32.753777+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37943099,"num_objects":9266,"num_object_clones":0,"num_object_copies":18532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11266,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2515},{"shards":"2","objects":5752}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11072","reported_seq":16100,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.668286+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:33.668286+0000","last_peered":"2024-09-16T04:53:33.668286+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:33.668286+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37380096,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":877,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7826,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2448,"num_bytes_recovered":10018816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8247},{"shards":"0,2","objects":877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11372","reported_seq":11512,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.607898+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:33.607898+0000","last_peered":"2024-09-16T04:53:33.607898+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:33.607898+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37044841,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8141,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5521},{"shards":"0,2","objects":2366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":591585527,"num_objects":144437,"num_object_clones":0,"num_object_copies":288874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":877,"num_objects_unfound":0,"num_objects_dirty":144437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142193,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5803,"num_bytes_recovered":23728128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1249112064,"data_stored":1249030861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751656,"kb_used_data":445068,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93620184,"statfs":{"total":96636764160,"available":95867068416,"internally_reserved":0,"allocated":455749632,"data_stored":455476577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593776,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609544,"kb_used_data":334904,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93762296,"statfs":{"total":96636764160,"available":96012591104,"internally_reserved":0,"allocated":342941696,"data_stored":342606019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":281210542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":749536,"kb_used_data":443020,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93622304,"statfs":{"total":96636764160,"available":95869239296,"internally_reserved":0,"allocated":453652480,"data_stored":453383497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,1,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":452104192,"data_stored":452100713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":342528000,"data_stored":342457827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454479872,"data_stored":454472321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:35.560 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:36.371 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:36.373 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:36.430 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-16T04:53:34.378109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591585546,"num_objects":144439,"num_object_clones":0,"num_object_copies":288878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":877,"num_objects_unfound":0,"num_objects_dirty":144439,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142195,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5804,"num_bytes_recovered":23728147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110736,"kb_used_data":1222992,"kb_used_omap":12,"kb_used_meta":887667,"kb_avail":281004784,"statfs":{"total":289910292480,"available":287748898816,"internally_reserved":0,"allocated":1252343808,"data_stored":1251466093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":908971009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,4,4,2,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27922432,"num_objects":-6817,"num_object_clones":0,"num_object_copies":-13634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-389,"num_objects_unfound":0,"num_objects_dirty":-6817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6817,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":1593344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004928"},"pg_stats":[{"pgid":"2.f","version":"47'10897","reported_seq":10976,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.572045+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:33.572045+0000","last_peered":"2024-09-16T04:53:33.572045+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:33.572045+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36212736,"num_objects":8841,"num_object_clones":0,"num_object_copies":17682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10897,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5500},{"shards":"0,2","objects":2312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11003","reported_seq":11120,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:32.754840+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:32.754840+0000","last_peered":"2024-09-16T04:53:32.754840+0000","last_clean":"2024-09-16T04:53:32.754840+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:32.754840+0000","last_undegraded":"2024-09-16T04:53:32.754840+0000","last_fullsized":"2024-09-16T04:53:32.754840+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36827136,"num_objects":8991,"num_object_clones":0,"num_object_copies":17982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7972,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10899","reported_seq":11039,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.580284+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:32.580284+0000","last_peered":"2024-09-16T04:53:32.580284+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:32.580284+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36691992,"num_objects":8959,"num_object_clones":0,"num_object_copies":17918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7638,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2351},{"shards":"2","objects":5640}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11170","reported_seq":11241,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:32.743336+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:32.743336+0000","last_peered":"2024-09-16T04:53:32.743336+0000","last_clean":"2024-09-16T04:53:32.743336+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:32.743336+0000","last_undegraded":"2024-09-16T04:53:32.743336+0000","last_fullsized":"2024-09-16T04:53:32.743336+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37576704,"num_objects":9174,"num_object_clones":0,"num_object_copies":18348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11170,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11103","reported_seq":11322,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:33.667259+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:33.667259+0000","last_peered":"2024-09-16T04:53:33.667259+0000","last_clean":"2024-09-16T04:53:33.667259+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:33.667259+0000","last_undegraded":"2024-09-16T04:53:33.667259+0000","last_fullsized":"2024-09-16T04:53:33.667259+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37031936,"num_objects":9041,"num_object_clones":0,"num_object_copies":18082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8135,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11211","reported_seq":11345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.563016+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:33.563016+0000","last_peered":"2024-09-16T04:53:33.563016+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:33.563016+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37281792,"num_objects":9102,"num_object_clones":0,"num_object_copies":18204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8204,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5646},{"shards":"0,2","objects":2404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10973","reported_seq":11122,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.601236+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:33.601236+0000","last_peered":"2024-09-16T04:53:33.601236+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:33.601236+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36401152,"num_objects":8887,"num_object_clones":0,"num_object_copies":17774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7628,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5488},{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11015","reported_seq":11231,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:33.651407+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:33.651407+0000","last_peered":"2024-09-16T04:53:33.651407+0000","last_clean":"2024-09-16T04:53:33.651407+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:33.651407+0000","last_undegraded":"2024-09-16T04:53:33.651407+0000","last_fullsized":"2024-09-16T04:53:33.651407+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36868096,"num_objects":9001,"num_object_clones":0,"num_object_copies":18002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8085,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11065","reported_seq":11292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.714659+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:32.714659+0000","last_peered":"2024-09-16T04:53:32.714659+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:32.714659+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36929536,"num_objects":9016,"num_object_clones":0,"num_object_copies":18032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7816,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2313},{"shards":"2","objects":5695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11194","reported_seq":11324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.532774+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:33.532774+0000","last_peered":"2024-09-16T04:53:33.532774+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:33.532774+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36962304,"num_objects":9024,"num_object_clones":0,"num_object_copies":18048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7849,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5559},{"shards":"0,2","objects":2384}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10965","reported_seq":11206,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.754550+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:32.754550+0000","last_peered":"2024-09-16T04:53:32.754550+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:32.754550+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36922578,"num_objects":9016,"num_object_clones":0,"num_object_copies":18032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10937,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2336},{"shards":"2","objects":5720}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10971","reported_seq":17828,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:32.665877+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:32.665877+0000","last_peered":"2024-09-16T04:53:32.665877+0000","last_clean":"2024-09-16T04:53:32.665877+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:32.665877+0000","last_undegraded":"2024-09-16T04:53:32.665877+0000","last_fullsized":"2024-09-16T04:53:32.665877+0000","mapping_epoch":46,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36892672,"num_objects":9007,"num_object_clones":0,"num_object_copies":18014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7694,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10936","reported_seq":11034,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.753631+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:32.753631+0000","last_peered":"2024-09-16T04:53:32.753631+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:32.753631+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36618857,"num_objects":8941,"num_object_clones":0,"num_object_copies":17882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10935,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2410},{"shards":"2","objects":5534}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11269","reported_seq":11370,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.753777+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:32.753777+0000","last_peered":"2024-09-16T04:53:32.753777+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:32.753777+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37943099,"num_objects":9266,"num_object_clones":0,"num_object_copies":18532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11266,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2515},{"shards":"2","objects":5752}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11072","reported_seq":16100,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.668286+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:33.668286+0000","last_peered":"2024-09-16T04:53:33.668286+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:33.668286+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37380096,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":877,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7826,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2448,"num_bytes_recovered":10018816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8247},{"shards":"0,2","objects":877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11372","reported_seq":11512,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.607898+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:33.607898+0000","last_peered":"2024-09-16T04:53:33.607898+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:33.607898+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37044841,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8141,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5521},{"shards":"0,2","objects":2366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":591585527,"num_objects":144437,"num_object_clones":0,"num_object_copies":288874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":877,"num_objects_unfound":0,"num_objects_dirty":144437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142193,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5803,"num_bytes_recovered":23728128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1249112064,"data_stored":1249030861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751656,"kb_used_data":445068,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93620184,"statfs":{"total":96636764160,"available":95867068416,"internally_reserved":0,"allocated":455749632,"data_stored":455476577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593776,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609544,"kb_used_data":334904,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93762296,"statfs":{"total":96636764160,"available":96012591104,"internally_reserved":0,"allocated":342941696,"data_stored":342606019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":281210542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":749536,"kb_used_data":443020,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93622304,"statfs":{"total":96636764160,"available":95869239296,"internally_reserved":0,"allocated":453652480,"data_stored":453383497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,1,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":452104192,"data_stored":452100713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":342528000,"data_stored":342457827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454479872,"data_stored":454472321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:36.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:53:36.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'10897", 'reported_seq': 10976, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:33.572045+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:53:33.572045+0000', 'last_peered': '2024-09-16T04:53:33.572045+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:53:33.572045+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36212736, 'num_objects': 8841, 'num_object_clones': 0, 'num_object_copies': 17682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8841, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10897, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5500}, {'shards': '0,2', 'objects': 2312}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:36.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:53:36.432 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'10899", 'reported_seq': 11039, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:32.580284+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:53:32.580284+0000', 'last_peered': '2024-09-16T04:53:32.580284+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:53:32.580284+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36691992, 'num_objects': 8959, 'num_object_clones': 0, 'num_object_copies': 17918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8959, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7638, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2351}, {'shards': '2', 'objects': 5640}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:36.433 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:53:36.433 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'11211", 'reported_seq': 11345, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:33.563016+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:53:33.563016+0000', 'last_peered': '2024-09-16T04:53:33.563016+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:53:33.563016+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37281792, 'num_objects': 9102, 'num_object_clones': 0, 'num_object_copies': 18204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9102, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8204, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5646}, {'shards': '0,2', 'objects': 2404}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:36.433 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:53:36.433 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'10973", 'reported_seq': 11122, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:33.601236+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:53:33.601236+0000', 'last_peered': '2024-09-16T04:53:33.601236+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:53:33.601236+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36401152, 'num_objects': 8887, 'num_object_clones': 0, 'num_object_copies': 17774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7628, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5488}, {'shards': '0,2', 'objects': 2359}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:36.433 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:53:36.433 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'11065", 'reported_seq': 11292, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:32.714659+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:53:32.714659+0000', 'last_peered': '2024-09-16T04:53:32.714659+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:53:32.714659+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36929536, 'num_objects': 9016, 'num_object_clones': 0, 'num_object_copies': 18032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7816, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2313}, {'shards': '2', 'objects': 5695}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:36.433 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:53:36.433 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'11194", 'reported_seq': 11324, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:33.532774+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:53:33.532774+0000', 'last_peered': '2024-09-16T04:53:33.532774+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:53:33.532774+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36962304, 'num_objects': 9024, 'num_object_clones': 0, 'num_object_copies': 18048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7849, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5559}, {'shards': '0,2', 'objects': 2384}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:36.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:53:36.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'10965", 'reported_seq': 11206, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:32.754550+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:53:32.754550+0000', 'last_peered': '2024-09-16T04:53:32.754550+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:53:32.754550+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36922578, 'num_objects': 9016, 'num_object_clones': 0, 'num_object_copies': 18032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10937, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2336}, {'shards': '2', 'objects': 5720}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:36.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:53:36.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'10936", 'reported_seq': 11034, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:32.753631+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:53:32.753631+0000', 'last_peered': '2024-09-16T04:53:32.753631+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:53:32.753631+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36618857, 'num_objects': 8941, 'num_object_clones': 0, 'num_object_copies': 17882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8941, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10935, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2410}, {'shards': '2', 'objects': 5534}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:36.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:53:36.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'11269", 'reported_seq': 11370, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:32.753777+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:53:32.753777+0000', 'last_peered': '2024-09-16T04:53:32.753777+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:53:32.753777+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37943099, 'num_objects': 9266, 'num_object_clones': 0, 'num_object_copies': 18532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9266, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11266, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2515}, {'shards': '2', 'objects': 5752}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:36.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:53:36.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'11072", 'reported_seq': 16100, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:33.668286+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:53:33.668286+0000', 'last_peered': '2024-09-16T04:53:33.668286+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:53:33.668286+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37380096, 'num_objects': 9126, 'num_object_clones': 0, 'num_object_copies': 18252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 877, 'num_objects_unfound': 0, 'num_objects_dirty': 9126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7826, 'num_write_kb': 27420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2448, 'num_bytes_recovered': 10018816, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8247}, {'shards': '0,2', 'objects': 877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:36.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:53:36.435 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'11372", 'reported_seq': 11512, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:33.607898+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:53:33.607898+0000', 'last_peered': '2024-09-16T04:53:33.607898+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:53:33.607898+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37044841, 'num_objects': 9045, 'num_object_clones': 0, 'num_object_copies': 18090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8141, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5521}, {'shards': '0,2', 'objects': 2366}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:36.435 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:53:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:37.295 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/288878 objects degraded (11.791%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":365,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":144439,"data_bytes":591585546,"bytes_used":2161393664,"bytes_avail":287748898816,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":288878,"degraded_ratio":0.11791136742846461,"misplaced_objects":877,"misplaced_total":288878,"misplaced_ratio":0.0030358836602302703,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":93145,"recovering_keys_per_sec":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":378},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:53:37.397 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:53:37.397 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:38.359 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:38.359 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:38.419 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-16T04:53:36.378612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591585546,"num_objects":144439,"num_object_clones":0,"num_object_copies":288878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":877,"num_objects_unfound":0,"num_objects_dirty":144439,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142195,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5804,"num_bytes_recovered":23728147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2110736,"kb_used_data":1222992,"kb_used_omap":12,"kb_used_meta":887667,"kb_avail":281004784,"statfs":{"total":289910292480,"available":287748898816,"internally_reserved":0,"allocated":1252343808,"data_stored":1251466093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":908971009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,4,4,2,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18632704,"num_objects":-4549,"num_object_clones":0,"num_object_copies":-9098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-273,"num_objects_unfound":0,"num_objects_dirty":-4549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4549,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004951"},"pg_stats":[{"pgid":"2.f","version":"47'10897","reported_seq":10976,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.572045+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:33.572045+0000","last_peered":"2024-09-16T04:53:33.572045+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:33.572045+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36212736,"num_objects":8841,"num_object_clones":0,"num_object_copies":17682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10897,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5500},{"shards":"0,2","objects":2312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11003","reported_seq":11120,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:32.754840+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:32.754840+0000","last_peered":"2024-09-16T04:53:32.754840+0000","last_clean":"2024-09-16T04:53:32.754840+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:32.754840+0000","last_undegraded":"2024-09-16T04:53:32.754840+0000","last_fullsized":"2024-09-16T04:53:32.754840+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36827136,"num_objects":8991,"num_object_clones":0,"num_object_copies":17982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7972,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'10899","reported_seq":11039,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.580284+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:32.580284+0000","last_peered":"2024-09-16T04:53:32.580284+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:32.580284+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36691992,"num_objects":8959,"num_object_clones":0,"num_object_copies":17918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7638,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2351},{"shards":"2","objects":5640}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11170","reported_seq":11241,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:32.743336+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:32.743336+0000","last_peered":"2024-09-16T04:53:32.743336+0000","last_clean":"2024-09-16T04:53:32.743336+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:32.743336+0000","last_undegraded":"2024-09-16T04:53:32.743336+0000","last_fullsized":"2024-09-16T04:53:32.743336+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37576704,"num_objects":9174,"num_object_clones":0,"num_object_copies":18348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11170,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11103","reported_seq":11322,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:33.667259+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:33.667259+0000","last_peered":"2024-09-16T04:53:33.667259+0000","last_clean":"2024-09-16T04:53:33.667259+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:33.667259+0000","last_undegraded":"2024-09-16T04:53:33.667259+0000","last_fullsized":"2024-09-16T04:53:33.667259+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37031936,"num_objects":9041,"num_object_clones":0,"num_object_copies":18082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8135,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11211","reported_seq":11345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.563016+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:33.563016+0000","last_peered":"2024-09-16T04:53:33.563016+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:33.563016+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37281792,"num_objects":9102,"num_object_clones":0,"num_object_copies":18204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8204,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5646},{"shards":"0,2","objects":2404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'10973","reported_seq":11122,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.601236+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:33.601236+0000","last_peered":"2024-09-16T04:53:33.601236+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:33.601236+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36401152,"num_objects":8887,"num_object_clones":0,"num_object_copies":17774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7628,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5488},{"shards":"0,2","objects":2359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11015","reported_seq":11231,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:33.651407+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:33.651407+0000","last_peered":"2024-09-16T04:53:33.651407+0000","last_clean":"2024-09-16T04:53:33.651407+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:33.651407+0000","last_undegraded":"2024-09-16T04:53:33.651407+0000","last_fullsized":"2024-09-16T04:53:33.651407+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36868096,"num_objects":9001,"num_object_clones":0,"num_object_copies":18002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8085,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149274+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:33.149274+0000","last_peered":"2024-09-16T04:51:33.149274+0000","last_clean":"2024-09-16T04:51:33.149274+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:33.149274+0000","last_undegraded":"2024-09-16T04:51:33.149274+0000","last_fullsized":"2024-09-16T04:51:33.149274+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11065","reported_seq":11292,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.714659+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:32.714659+0000","last_peered":"2024-09-16T04:53:32.714659+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:32.714659+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36929536,"num_objects":9016,"num_object_clones":0,"num_object_copies":18032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7816,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2313},{"shards":"2","objects":5695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11194","reported_seq":11324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.532774+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:33.532774+0000","last_peered":"2024-09-16T04:53:33.532774+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:33.532774+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36962304,"num_objects":9024,"num_object_clones":0,"num_object_copies":18048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7849,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5559},{"shards":"0,2","objects":2384}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'10965","reported_seq":11206,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.754550+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:32.754550+0000","last_peered":"2024-09-16T04:53:32.754550+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:32.754550+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36922578,"num_objects":9016,"num_object_clones":0,"num_object_copies":18032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10937,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2336},{"shards":"2","objects":5720}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'10971","reported_seq":17828,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:32.665877+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:32.665877+0000","last_peered":"2024-09-16T04:53:32.665877+0000","last_clean":"2024-09-16T04:53:32.665877+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:32.665877+0000","last_undegraded":"2024-09-16T04:53:32.665877+0000","last_fullsized":"2024-09-16T04:53:32.665877+0000","mapping_epoch":46,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36892672,"num_objects":9007,"num_object_clones":0,"num_object_copies":18014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7694,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'10936","reported_seq":11034,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.753631+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:32.753631+0000","last_peered":"2024-09-16T04:53:32.753631+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:32.753631+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36618857,"num_objects":8941,"num_object_clones":0,"num_object_copies":17882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10935,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2410},{"shards":"2","objects":5534}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11269","reported_seq":11370,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:32.753777+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:32.753777+0000","last_peered":"2024-09-16T04:53:32.753777+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:32.753777+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37943099,"num_objects":9266,"num_object_clones":0,"num_object_copies":18532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11266,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2515},{"shards":"2","objects":5752}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11072","reported_seq":16100,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.668286+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:33.668286+0000","last_peered":"2024-09-16T04:53:33.668286+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:33.668286+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37380096,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":877,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7826,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2448,"num_bytes_recovered":10018816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8247},{"shards":"0,2","objects":877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11372","reported_seq":11512,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:33.607898+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:33.607898+0000","last_peered":"2024-09-16T04:53:33.607898+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:33.607898+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37044841,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8141,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5521},{"shards":"0,2","objects":2366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":591585527,"num_objects":144437,"num_object_clones":0,"num_object_copies":288874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":877,"num_objects_unfound":0,"num_objects_dirty":144437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142193,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5803,"num_bytes_recovered":23728128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1249112064,"data_stored":1249030861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160915,"ondisk_log_size":160915,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751656,"kb_used_data":445068,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93620184,"statfs":{"total":96636764160,"available":95867068416,"internally_reserved":0,"allocated":455749632,"data_stored":455476577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593776,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609544,"kb_used_data":334904,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93762296,"statfs":{"total":96636764160,"available":96012591104,"internally_reserved":0,"allocated":342941696,"data_stored":342606019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":281210542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":749536,"kb_used_data":443020,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93622304,"statfs":{"total":96636764160,"available":95869239296,"internally_reserved":0,"allocated":453652480,"data_stored":453383497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,1,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":452104192,"data_stored":452100713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":342528000,"data_stored":342457827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":454479872,"data_stored":454472321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:41.421 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:42.329 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-16T04:53:40.379533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":582213898,"num_objects":142151,"num_object_clones":0,"num_object_copies":284302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":768,"num_objects_unfound":0,"num_objects_dirty":142151,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":144483,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5913,"num_bytes_recovered":24174611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160606,"ondisk_log_size":160606,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2098824,"kb_used_data":1211092,"kb_used_omap":12,"kb_used_meta":887667,"kb_avail":281016696,"statfs":{"total":289910292480,"available":287761096704,"internally_reserved":0,"allocated":1240158208,"data_stored":1239280493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":908971009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,6,12,13,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23343104,"num_objects":-5699,"num_object_clones":0,"num_object_copies":-11398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-382,"num_objects_unfound":0,"num_objects_dirty":-5699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5699,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":1564672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004909"},"pg_stats":[{"pgid":"2.f","version":"47'11032","reported_seq":11112,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:38.527583+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:38.527583+0000","last_peered":"2024-09-16T04:53:38.527583+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:38.527583+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35659776,"num_objects":8706,"num_object_clones":0,"num_object_copies":17412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11032,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5365},{"shards":"0,2","objects":2177}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11139","reported_seq":11256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:37.719171+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:37.719171+0000","last_peered":"2024-09-16T04:53:37.719171+0000","last_clean":"2024-09-16T04:53:37.719171+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:37.719171+0000","last_undegraded":"2024-09-16T04:53:37.719171+0000","last_fullsized":"2024-09-16T04:53:37.719171+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36270080,"num_objects":8855,"num_object_clones":0,"num_object_copies":17710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8108,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'11042","reported_seq":11181,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:37.741521+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:37.741521+0000","last_peered":"2024-09-16T04:53:37.741521+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:37.741521+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36106264,"num_objects":8816,"num_object_clones":0,"num_object_copies":17632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7781,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2208},{"shards":"2","objects":5497}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11330","reported_seq":11401,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:37.700145+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:37.700145+0000","last_peered":"2024-09-16T04:53:37.700145+0000","last_clean":"2024-09-16T04:53:37.700145+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:37.700145+0000","last_undegraded":"2024-09-16T04:53:37.700145+0000","last_fullsized":"2024-09-16T04:53:37.700145+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36921344,"num_objects":9014,"num_object_clones":0,"num_object_copies":18028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11330,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11242","reported_seq":11461,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:38.613206+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:38.613206+0000","last_peered":"2024-09-16T04:53:38.613206+0000","last_clean":"2024-09-16T04:53:38.613206+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:38.613206+0000","last_undegraded":"2024-09-16T04:53:38.613206+0000","last_fullsized":"2024-09-16T04:53:38.613206+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36462592,"num_objects":8902,"num_object_clones":0,"num_object_copies":17804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8274,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11346","reported_seq":11480,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:38.665983+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:38.665983+0000","last_peered":"2024-09-16T04:53:38.665983+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:38.665983+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36728832,"num_objects":8967,"num_object_clones":0,"num_object_copies":17934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8339,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5511},{"shards":"0,2","objects":2269}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11101","reported_seq":11250,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:38.609546+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:38.609546+0000","last_peered":"2024-09-16T04:53:38.609546+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:38.609546+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35876864,"num_objects":8759,"num_object_clones":0,"num_object_copies":17518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7756,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5360},{"shards":"0,2","objects":2231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11158","reported_seq":11374,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:38.675490+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:38.675490+0000","last_peered":"2024-09-16T04:53:38.675490+0000","last_clean":"2024-09-16T04:53:38.675490+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:38.675490+0000","last_undegraded":"2024-09-16T04:53:38.675490+0000","last_fullsized":"2024-09-16T04:53:38.675490+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36282368,"num_objects":8858,"num_object_clones":0,"num_object_copies":17716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8228,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:35.473233+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:35.473233+0000","last_peered":"2024-09-16T04:51:35.473233+0000","last_clean":"2024-09-16T04:51:35.473233+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:35.473233+0000","last_undegraded":"2024-09-16T04:51:35.473233+0000","last_fullsized":"2024-09-16T04:51:35.473233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11193","reported_seq":11422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:37.813104+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:37.813104+0000","last_peered":"2024-09-16T04:53:37.813104+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:37.813104+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36405248,"num_objects":8888,"num_object_clones":0,"num_object_copies":17776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7944,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2185},{"shards":"2","objects":5567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11341","reported_seq":11470,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:38.613379+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:38.613379+0000","last_peered":"2024-09-16T04:53:38.613379+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:38.613379+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36360192,"num_objects":8877,"num_object_clones":0,"num_object_copies":17754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7996,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5412},{"shards":"0,2","objects":2237}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11114","reported_seq":11352,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:37.609053+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:37.609053+0000","last_peered":"2024-09-16T04:53:37.609053+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:37.609053+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36312274,"num_objects":8867,"num_object_clones":0,"num_object_copies":17734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11086,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2187},{"shards":"2","objects":5571}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'11111","reported_seq":17968,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:37.741732+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:37.741732+0000","last_peered":"2024-09-16T04:53:37.741732+0000","last_clean":"2024-09-16T04:53:37.741732+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:37.741732+0000","last_undegraded":"2024-09-16T04:53:37.741732+0000","last_fullsized":"2024-09-16T04:53:37.741732+0000","mapping_epoch":46,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36319232,"num_objects":8867,"num_object_clones":0,"num_object_copies":17734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7834,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'11078","reported_seq":11176,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:37.753496+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:37.753496+0000","last_peered":"2024-09-16T04:53:37.753496+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:37.753496+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36037225,"num_objects":8799,"num_object_clones":0,"num_object_copies":17598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11077,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2268},{"shards":"2","objects":5392}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11438","reported_seq":11540,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:37.769324+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:37.769324+0000","last_peered":"2024-09-16T04:53:37.769324+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:37.769324+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37250875,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11435,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2346},{"shards":"2","objects":5583}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11218","reported_seq":16465,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:38.703671+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:38.703671+0000","last_peered":"2024-09-16T04:53:38.703671+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:38.703671+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36782080,"num_objects":8980,"num_object_clones":0,"num_object_copies":17960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":768,"num_objects_unfound":0,"num_objects_dirty":8980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7972,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2557,"num_bytes_recovered":10465280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8210},{"shards":"0,2","objects":768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11520","reported_seq":11661,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:38.652573+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:38.652573+0000","last_peered":"2024-09-16T04:53:38.652573+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:38.652573+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36438633,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8289,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5373},{"shards":"0,2","objects":2218}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":582213879,"num_objects":142149,"num_object_clones":0,"num_object_copies":284298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":768,"num_objects_unfound":0,"num_objects_dirty":142149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":144481,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5912,"num_bytes_recovered":24174592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1236848640,"data_stored":1236767437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160603,"ondisk_log_size":160603,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754280,"kb_used_data":439380,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93617560,"statfs":{"total":96636764160,"available":95864381440,"internally_reserved":0,"allocated":449925120,"data_stored":449652065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":322432681},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4,3,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593777,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596492,"kb_used_data":334332,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93775348,"statfs":{"total":96636764160,"available":96025956352,"internally_reserved":0,"allocated":342355968,"data_stored":342020291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":748052,"kb_used_data":437380,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93623788,"statfs":{"total":96636764160,"available":95870758912,"internally_reserved":0,"allocated":447877120,"data_stored":447608137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,0,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":446328832,"data_stored":446325353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":341942272,"data_stored":341872099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448577536,"data_stored":448569985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:42.330 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:43.081 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:43.081 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:43.140 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-16T04:53:42.380013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":582213898,"num_objects":142151,"num_object_clones":0,"num_object_copies":284302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":768,"num_objects_unfound":0,"num_objects_dirty":142151,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":144483,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5913,"num_bytes_recovered":24174611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160606,"ondisk_log_size":160606,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2106584,"kb_used_data":1210528,"kb_used_omap":12,"kb_used_meta":895987,"kb_avail":281008936,"statfs":{"total":289910292480,"available":287753150464,"internally_reserved":0,"allocated":1239580672,"data_stored":1238702957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":917490689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,6,6,13,7,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18636800,"num_objects":-4550,"num_object_clones":0,"num_object_copies":-9100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-247,"num_objects_unfound":0,"num_objects_dirty":-4550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4550,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":1011712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002859"},"pg_stats":[{"pgid":"2.f","version":"47'11032","reported_seq":11112,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:38.527583+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:38.527583+0000","last_peered":"2024-09-16T04:53:38.527583+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:38.527583+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35659776,"num_objects":8706,"num_object_clones":0,"num_object_copies":17412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11032,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5365},{"shards":"0,2","objects":2177}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11139","reported_seq":11256,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:37.719171+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:37.719171+0000","last_peered":"2024-09-16T04:53:37.719171+0000","last_clean":"2024-09-16T04:53:37.719171+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:37.719171+0000","last_undegraded":"2024-09-16T04:53:37.719171+0000","last_fullsized":"2024-09-16T04:53:37.719171+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36270080,"num_objects":8855,"num_object_clones":0,"num_object_copies":17710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8108,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'11042","reported_seq":11181,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:37.741521+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:37.741521+0000","last_peered":"2024-09-16T04:53:37.741521+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:37.741521+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36106264,"num_objects":8816,"num_object_clones":0,"num_object_copies":17632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8816,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7781,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2208},{"shards":"2","objects":5497}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11330","reported_seq":11401,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:37.700145+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:37.700145+0000","last_peered":"2024-09-16T04:53:37.700145+0000","last_clean":"2024-09-16T04:53:37.700145+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:37.700145+0000","last_undegraded":"2024-09-16T04:53:37.700145+0000","last_fullsized":"2024-09-16T04:53:37.700145+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36921344,"num_objects":9014,"num_object_clones":0,"num_object_copies":18028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11330,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11242","reported_seq":11461,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:38.613206+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:38.613206+0000","last_peered":"2024-09-16T04:53:38.613206+0000","last_clean":"2024-09-16T04:53:38.613206+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:38.613206+0000","last_undegraded":"2024-09-16T04:53:38.613206+0000","last_fullsized":"2024-09-16T04:53:38.613206+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36462592,"num_objects":8902,"num_object_clones":0,"num_object_copies":17804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8274,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11346","reported_seq":11480,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:38.665983+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:38.665983+0000","last_peered":"2024-09-16T04:53:38.665983+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:38.665983+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36728832,"num_objects":8967,"num_object_clones":0,"num_object_copies":17934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8339,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5511},{"shards":"0,2","objects":2269}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11101","reported_seq":11250,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:38.609546+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:38.609546+0000","last_peered":"2024-09-16T04:53:38.609546+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:38.609546+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35876864,"num_objects":8759,"num_object_clones":0,"num_object_copies":17518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7756,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5360},{"shards":"0,2","objects":2231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11158","reported_seq":11374,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:38.675490+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:38.675490+0000","last_peered":"2024-09-16T04:53:38.675490+0000","last_clean":"2024-09-16T04:53:38.675490+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:38.675490+0000","last_undegraded":"2024-09-16T04:53:38.675490+0000","last_fullsized":"2024-09-16T04:53:38.675490+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36282368,"num_objects":8858,"num_object_clones":0,"num_object_copies":17716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8228,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:35.473233+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:35.473233+0000","last_peered":"2024-09-16T04:51:35.473233+0000","last_clean":"2024-09-16T04:51:35.473233+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:35.473233+0000","last_undegraded":"2024-09-16T04:51:35.473233+0000","last_fullsized":"2024-09-16T04:51:35.473233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11193","reported_seq":11422,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:37.813104+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:37.813104+0000","last_peered":"2024-09-16T04:53:37.813104+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:37.813104+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36405248,"num_objects":8888,"num_object_clones":0,"num_object_copies":17776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7944,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2185},{"shards":"2","objects":5567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11341","reported_seq":11470,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:38.613379+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:38.613379+0000","last_peered":"2024-09-16T04:53:38.613379+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:38.613379+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36360192,"num_objects":8877,"num_object_clones":0,"num_object_copies":17754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7996,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5412},{"shards":"0,2","objects":2237}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11114","reported_seq":11352,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:37.609053+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:37.609053+0000","last_peered":"2024-09-16T04:53:37.609053+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:37.609053+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36312274,"num_objects":8867,"num_object_clones":0,"num_object_copies":17734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11086,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2187},{"shards":"2","objects":5571}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'11111","reported_seq":17968,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:37.741732+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:37.741732+0000","last_peered":"2024-09-16T04:53:37.741732+0000","last_clean":"2024-09-16T04:53:37.741732+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:37.741732+0000","last_undegraded":"2024-09-16T04:53:37.741732+0000","last_fullsized":"2024-09-16T04:53:37.741732+0000","mapping_epoch":46,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36319232,"num_objects":8867,"num_object_clones":0,"num_object_copies":17734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7834,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'11078","reported_seq":11176,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:37.753496+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:37.753496+0000","last_peered":"2024-09-16T04:53:37.753496+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:37.753496+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36037225,"num_objects":8799,"num_object_clones":0,"num_object_copies":17598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11077,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2268},{"shards":"2","objects":5392}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11438","reported_seq":11540,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:37.769324+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:37.769324+0000","last_peered":"2024-09-16T04:53:37.769324+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:37.769324+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37250875,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11435,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2346},{"shards":"2","objects":5583}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11218","reported_seq":16465,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:38.703671+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:38.703671+0000","last_peered":"2024-09-16T04:53:38.703671+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:38.703671+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36782080,"num_objects":8980,"num_object_clones":0,"num_object_copies":17960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":768,"num_objects_unfound":0,"num_objects_dirty":8980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7972,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2557,"num_bytes_recovered":10465280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8210},{"shards":"0,2","objects":768}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11520","reported_seq":11661,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:38.652573+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:38.652573+0000","last_peered":"2024-09-16T04:53:38.652573+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:38.652573+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36438633,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8289,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5373},{"shards":"0,2","objects":2218}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":582213879,"num_objects":142149,"num_object_clones":0,"num_object_copies":284298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":768,"num_objects_unfound":0,"num_objects_dirty":142149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":144481,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5912,"num_bytes_recovered":24174592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1236246528,"data_stored":1236165325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160603,"ondisk_log_size":160603,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754280,"kb_used_data":439380,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93617560,"statfs":{"total":96636764160,"available":95864381440,"internally_reserved":0,"allocated":449925120,"data_stored":449652065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":322432681},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4,3,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593778,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604252,"kb_used_data":333768,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93767588,"statfs":{"total":96636764160,"available":96018010112,"internally_reserved":0,"allocated":341778432,"data_stored":341442755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":748052,"kb_used_data":437380,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93623788,"statfs":{"total":96636764160,"available":95870758912,"internally_reserved":0,"allocated":447877120,"data_stored":447608137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,0,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":446328832,"data_stored":446325353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":341340160,"data_stored":341269987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448577536,"data_stored":448569985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:43.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:53:43.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'11032", 'reported_seq': 11112, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:38.527583+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:53:38.527583+0000', 'last_peered': '2024-09-16T04:53:38.527583+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:53:38.527583+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35659776, 'num_objects': 8706, 'num_object_clones': 0, 'num_object_copies': 17412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8706, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11032, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5365}, {'shards': '0,2', 'objects': 2177}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:43.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:53:43.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'11042", 'reported_seq': 11181, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:37.741521+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:53:37.741521+0000', 'last_peered': '2024-09-16T04:53:37.741521+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:53:37.741521+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36106264, 'num_objects': 8816, 'num_object_clones': 0, 'num_object_copies': 17632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8816, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7781, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2208}, {'shards': '2', 'objects': 5497}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:43.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:53:43.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'11346", 'reported_seq': 11480, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:38.665983+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:53:38.665983+0000', 'last_peered': '2024-09-16T04:53:38.665983+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:53:38.665983+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36728832, 'num_objects': 8967, 'num_object_clones': 0, 'num_object_copies': 17934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8967, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8339, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5511}, {'shards': '0,2', 'objects': 2269}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:43.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:53:43.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'11101", 'reported_seq': 11250, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:38.609546+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:53:38.609546+0000', 'last_peered': '2024-09-16T04:53:38.609546+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:53:38.609546+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35876864, 'num_objects': 8759, 'num_object_clones': 0, 'num_object_copies': 17518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8759, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7756, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5360}, {'shards': '0,2', 'objects': 2231}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:43.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:53:43.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'11193", 'reported_seq': 11422, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:37.813104+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:53:37.813104+0000', 'last_peered': '2024-09-16T04:53:37.813104+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:53:37.813104+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36405248, 'num_objects': 8888, 'num_object_clones': 0, 'num_object_copies': 17776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7944, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2185}, {'shards': '2', 'objects': 5567}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:43.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:53:43.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'11341", 'reported_seq': 11470, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:38.613379+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:53:38.613379+0000', 'last_peered': '2024-09-16T04:53:38.613379+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:53:38.613379+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36360192, 'num_objects': 8877, 'num_object_clones': 0, 'num_object_copies': 17754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8877, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7996, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5412}, {'shards': '0,2', 'objects': 2237}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:43.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:53:43.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'11114", 'reported_seq': 11352, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:37.609053+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:53:37.609053+0000', 'last_peered': '2024-09-16T04:53:37.609053+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:53:37.609053+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36312274, 'num_objects': 8867, 'num_object_clones': 0, 'num_object_copies': 17734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8867, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11086, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2187}, {'shards': '2', 'objects': 5571}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:43.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:53:43.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'11078", 'reported_seq': 11176, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:37.753496+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:53:37.753496+0000', 'last_peered': '2024-09-16T04:53:37.753496+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:53:37.753496+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36037225, 'num_objects': 8799, 'num_object_clones': 0, 'num_object_copies': 17598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8799, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11077, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2268}, {'shards': '2', 'objects': 5392}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:43.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:53:43.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'11438", 'reported_seq': 11540, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:37.769324+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:53:37.769324+0000', 'last_peered': '2024-09-16T04:53:37.769324+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:53:37.769324+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37250875, 'num_objects': 9097, 'num_object_clones': 0, 'num_object_copies': 18194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9097, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11435, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2346}, {'shards': '2', 'objects': 5583}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:43.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:53:43.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'11218", 'reported_seq': 16465, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:38.703671+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:53:38.703671+0000', 'last_peered': '2024-09-16T04:53:38.703671+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:53:38.703671+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36782080, 'num_objects': 8980, 'num_object_clones': 0, 'num_object_copies': 17960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 768, 'num_objects_unfound': 0, 'num_objects_dirty': 8980, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7972, 'num_write_kb': 27420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2557, 'num_bytes_recovered': 10465280, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8210}, {'shards': '0,2', 'objects': 768}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:43.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:53:43.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'11520", 'reported_seq': 11661, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:38.652573+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:53:38.652573+0000', 'last_peered': '2024-09-16T04:53:38.652573+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:53:38.652573+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36438633, 'num_objects': 8897, 'num_object_clones': 0, 'num_object_copies': 17794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8897, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8289, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5373}, {'shards': '0,2', 'objects': 2218}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:43.144 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:53:44.132 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:44.132 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/284302 objects degraded (11.981%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":371,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":142151,"data_bytes":582213898,"bytes_used":2157142016,"bytes_avail":287753150464,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":284302,"degraded_ratio":0.11980921695943046,"misplaced_objects":768,"misplaced_total":284302,"misplaced_ratio":0.0027013527868252774,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":84289,"recovering_keys_per_sec":0,"num_objects_recovered":247,"num_bytes_recovered":1011712,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":379},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:53:44.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:53:44.175 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:44.883 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:45.004 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-16T04:53:44.380473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":572891402,"num_objects":139875,"num_object_clones":0,"num_object_copies":279750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":623,"num_objects_unfound":0,"num_objects_dirty":139875,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":146759,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6058,"num_bytes_recovered":24768531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161082,"ondisk_log_size":161082,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2074448,"kb_used_data":1199196,"kb_used_omap":12,"kb_used_meta":875187,"kb_avail":281041072,"statfs":{"total":289910292480,"available":287786057728,"internally_reserved":0,"allocated":1227976704,"data_stored":1227098989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":896191489},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,6,12,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27959296,"num_objects":-6826,"num_object_clones":0,"num_object_copies":-13652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-392,"num_objects_unfound":0,"num_objects_dirty":-6826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6826,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":1605632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002844"},"pg_stats":[{"pgid":"2.f","version":"47'11178","reported_seq":11258,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.660136+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:43.660136+0000","last_peered":"2024-09-16T04:53:43.660136+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:43.660136+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35061760,"num_objects":8560,"num_object_clones":0,"num_object_copies":17120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11178,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5219},{"shards":"0,2","objects":2031}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11278","reported_seq":11395,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:42.810220+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:42.810220+0000","last_peered":"2024-09-16T04:53:42.810220+0000","last_clean":"2024-09-16T04:53:42.810220+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:42.810220+0000","last_undegraded":"2024-09-16T04:53:42.810220+0000","last_fullsized":"2024-09-16T04:53:42.810220+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35700736,"num_objects":8716,"num_object_clones":0,"num_object_copies":17432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8247,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'11185","reported_seq":11325,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:42.715680+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:42.715680+0000","last_peered":"2024-09-16T04:53:42.715680+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:42.715680+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35520536,"num_objects":8673,"num_object_clones":0,"num_object_copies":17346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8673,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7924,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2065},{"shards":"2","objects":5354}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11475","reported_seq":11546,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:42.692547+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:42.692547+0000","last_peered":"2024-09-16T04:53:42.692547+0000","last_clean":"2024-09-16T04:53:42.692547+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:42.692547+0000","last_undegraded":"2024-09-16T04:53:42.692547+0000","last_fullsized":"2024-09-16T04:53:42.692547+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36327424,"num_objects":8869,"num_object_clones":0,"num_object_copies":17738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11475,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11369","reported_seq":11588,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:43.584304+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:43.584304+0000","last_peered":"2024-09-16T04:53:43.584304+0000","last_clean":"2024-09-16T04:53:43.584304+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:43.584304+0000","last_undegraded":"2024-09-16T04:53:43.584304+0000","last_fullsized":"2024-09-16T04:53:43.584304+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35942400,"num_objects":8775,"num_object_clones":0,"num_object_copies":17550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8401,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11489","reported_seq":11623,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.637429+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:43.637429+0000","last_peered":"2024-09-16T04:53:43.637429+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:43.637429+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36143104,"num_objects":8824,"num_object_clones":0,"num_object_copies":17648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8482,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5368},{"shards":"0,2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11261","reported_seq":11410,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.627406+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:43.627406+0000","last_peered":"2024-09-16T04:53:43.627406+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:43.627406+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35221504,"num_objects":8599,"num_object_clones":0,"num_object_copies":17198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7916,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5200},{"shards":"0,2","objects":2071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11290","reported_seq":11506,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:43.686570+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:43.686570+0000","last_peered":"2024-09-16T04:53:43.686570+0000","last_clean":"2024-09-16T04:53:43.686570+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:43.686570+0000","last_undegraded":"2024-09-16T04:53:43.686570+0000","last_fullsized":"2024-09-16T04:53:43.686570+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35741696,"num_objects":8726,"num_object_clones":0,"num_object_copies":17452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8360,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:35.473233+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:35.473233+0000","last_peered":"2024-09-16T04:51:35.473233+0000","last_clean":"2024-09-16T04:51:35.473233+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:35.473233+0000","last_undegraded":"2024-09-16T04:51:35.473233+0000","last_fullsized":"2024-09-16T04:51:35.473233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11337","reported_seq":11566,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:42.733652+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:42.733652+0000","last_peered":"2024-09-16T04:53:42.733652+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:42.733652+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35815424,"num_objects":8744,"num_object_clones":0,"num_object_copies":17488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8088,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2041},{"shards":"2","objects":5423}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11496","reported_seq":11625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.693689+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:43.693689+0000","last_peered":"2024-09-16T04:53:43.693689+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:43.693689+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35725312,"num_objects":8722,"num_object_clones":0,"num_object_copies":17444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8151,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5257},{"shards":"0,2","objects":2082}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11248","reported_seq":11489,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:42.809566+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:42.809566+0000","last_peered":"2024-09-16T04:53:42.809566+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:42.809566+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35763410,"num_objects":8733,"num_object_clones":0,"num_object_copies":17466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11220,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2053},{"shards":"2","objects":5437}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149466+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:33.149466+0000","last_peered":"2024-09-16T04:51:33.149466+0000","last_clean":"2024-09-16T04:51:33.149466+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:33.149466+0000","last_undegraded":"2024-09-16T04:51:33.149466+0000","last_fullsized":"2024-09-16T04:51:33.149466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'11227","reported_seq":18084,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:42.717468+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:42.717468+0000","last_peered":"2024-09-16T04:53:42.717468+0000","last_clean":"2024-09-16T04:53:42.717468+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:42.717468+0000","last_undegraded":"2024-09-16T04:53:42.717468+0000","last_fullsized":"2024-09-16T04:53:42.717468+0000","mapping_epoch":46,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35844096,"num_objects":8751,"num_object_clones":0,"num_object_copies":17502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7950,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'11240","reported_seq":11338,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:42.719529+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:42.719529+0000","last_peered":"2024-09-16T04:53:42.719529+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:42.719529+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35373673,"num_objects":8637,"num_object_clones":0,"num_object_copies":17274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11239,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2106},{"shards":"2","objects":5230}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149685+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:33.149685+0000","last_peered":"2024-09-16T04:51:33.149685+0000","last_clean":"2024-09-16T04:51:33.149685+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:33.149685+0000","last_undegraded":"2024-09-16T04:51:33.149685+0000","last_fullsized":"2024-09-16T04:51:33.149685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11583","reported_seq":11685,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:42.763482+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:42.763482+0000","last_peered":"2024-09-16T04:53:42.763482+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:42.763482+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36656955,"num_objects":8952,"num_object_clones":0,"num_object_copies":17904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11580,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2201},{"shards":"2","objects":5438}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11355","reported_seq":16892,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.644835+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:43.644835+0000","last_peered":"2024-09-16T04:53:43.644835+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:43.644835+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36220928,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":623,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8109,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2702,"num_bytes_recovered":11059200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8218},{"shards":"0,2","objects":623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11668","reported_seq":11809,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.656663+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:43.656663+0000","last_peered":"2024-09-16T04:53:43.656663+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:43.656663+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35832425,"num_objects":8749,"num_object_clones":0,"num_object_copies":17498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8437,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5225},{"shards":"0,2","objects":2070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":572891383,"num_objects":139873,"num_object_clones":0,"num_object_copies":279746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":623,"num_objects_unfound":0,"num_objects_dirty":139873,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":146757,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6057,"num_bytes_recovered":24768512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1224585216,"data_stored":1224504013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161079,"ondisk_log_size":161079,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736260,"kb_used_data":433844,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93635580,"statfs":{"total":96636764160,"available":95882833920,"internally_reserved":0,"allocated":444256256,"data_stored":443983201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593778,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604252,"kb_used_data":333768,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93767588,"statfs":{"total":96636764160,"available":96018010112,"internally_reserved":0,"allocated":341778432,"data_stored":341442755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":733936,"kb_used_data":431584,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93637904,"statfs":{"total":96636764160,"available":95885213696,"internally_reserved":0,"allocated":441942016,"data_stored":441673033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440397824,"data_stored":440394345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":341340160,"data_stored":341269987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":442847232,"data_stored":442839681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:48.006 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:48.971 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:48.977 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:49.023 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-16T04:53:48.381565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":568410378,"num_objects":138781,"num_object_clones":0,"num_object_copies":277562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":623,"num_objects_unfound":0,"num_objects_dirty":138781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147853,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6058,"num_bytes_recovered":24768531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160976,"ondisk_log_size":160976,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2094936,"kb_used_data":1193320,"kb_used_omap":12,"kb_used_meta":901555,"kb_avail":281020584,"statfs":{"total":289910292480,"available":287765078016,"internally_reserved":0,"allocated":1221959680,"data_stored":1221081965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":923192321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,6,16,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23175168,"num_objects":-5658,"num_object_clones":0,"num_object_copies":-11316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-254,"num_objects_unfound":0,"num_objects_dirty":-5658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5658,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002952"},"pg_stats":[{"pgid":"2.f","version":"47'11178","reported_seq":11258,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.660136+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:43.660136+0000","last_peered":"2024-09-16T04:53:43.660136+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:43.660136+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35061760,"num_objects":8560,"num_object_clones":0,"num_object_copies":17120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11178,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5219},{"shards":"0,2","objects":2031}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11429","reported_seq":11546,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:47.854117+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:47.854117+0000","last_peered":"2024-09-16T04:53:47.854117+0000","last_clean":"2024-09-16T04:53:47.854117+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:47.854117+0000","last_undegraded":"2024-09-16T04:53:47.854117+0000","last_fullsized":"2024-09-16T04:53:47.854117+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35082240,"num_objects":8565,"num_object_clones":0,"num_object_copies":17130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8398,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'11314","reported_seq":11454,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.822559+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:47.822559+0000","last_peered":"2024-09-16T04:53:47.822559+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:47.822559+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34992152,"num_objects":8544,"num_object_clones":0,"num_object_copies":17088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8053,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1936},{"shards":"2","objects":5225}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11615","reported_seq":11686,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:47.823850+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:47.823850+0000","last_peered":"2024-09-16T04:53:47.823850+0000","last_clean":"2024-09-16T04:53:47.823850+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:47.823850+0000","last_undegraded":"2024-09-16T04:53:47.823850+0000","last_fullsized":"2024-09-16T04:53:47.823850+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35753984,"num_objects":8729,"num_object_clones":0,"num_object_copies":17458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11615,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11369","reported_seq":11588,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:43.584304+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:43.584304+0000","last_peered":"2024-09-16T04:53:43.584304+0000","last_clean":"2024-09-16T04:53:43.584304+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:43.584304+0000","last_undegraded":"2024-09-16T04:53:43.584304+0000","last_fullsized":"2024-09-16T04:53:43.584304+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35942400,"num_objects":8775,"num_object_clones":0,"num_object_copies":17550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8401,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11489","reported_seq":11623,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.637429+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:43.637429+0000","last_peered":"2024-09-16T04:53:43.637429+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:43.637429+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36143104,"num_objects":8824,"num_object_clones":0,"num_object_copies":17648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8482,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5368},{"shards":"0,2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11261","reported_seq":11410,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.627406+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:43.627406+0000","last_peered":"2024-09-16T04:53:43.627406+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:43.627406+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35221504,"num_objects":8599,"num_object_clones":0,"num_object_copies":17198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7916,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5200},{"shards":"0,2","objects":2071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11290","reported_seq":11506,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:43.686570+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:43.686570+0000","last_peered":"2024-09-16T04:53:43.686570+0000","last_clean":"2024-09-16T04:53:43.686570+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:43.686570+0000","last_undegraded":"2024-09-16T04:53:43.686570+0000","last_fullsized":"2024-09-16T04:53:43.686570+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35741696,"num_objects":8726,"num_object_clones":0,"num_object_copies":17452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8360,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:35.473233+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:35.473233+0000","last_peered":"2024-09-16T04:51:35.473233+0000","last_clean":"2024-09-16T04:51:35.473233+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:35.473233+0000","last_undegraded":"2024-09-16T04:51:35.473233+0000","last_fullsized":"2024-09-16T04:51:35.473233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11472","reported_seq":11701,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.875030+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:47.875030+0000","last_peered":"2024-09-16T04:53:47.875030+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:47.875030+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35262464,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8223,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1906},{"shards":"2","objects":5288}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11496","reported_seq":11625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.693689+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:43.693689+0000","last_peered":"2024-09-16T04:53:43.693689+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:43.693689+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35725312,"num_objects":8722,"num_object_clones":0,"num_object_copies":17444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8151,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5257},{"shards":"0,2","objects":2082}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11369","reported_seq":11612,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.860224+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:47.860224+0000","last_peered":"2024-09-16T04:53:47.860224+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:47.860224+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35267794,"num_objects":8612,"num_object_clones":0,"num_object_copies":17224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11341,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1932},{"shards":"2","objects":5316}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550553+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:40.550553+0000","last_peered":"2024-09-16T04:51:40.550553+0000","last_clean":"2024-09-16T04:51:40.550553+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:40.550553+0000","last_undegraded":"2024-09-16T04:51:40.550553+0000","last_fullsized":"2024-09-16T04:51:40.550553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'11361","reported_seq":18218,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:47.778758+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:47.778758+0000","last_peered":"2024-09-16T04:53:47.778758+0000","last_clean":"2024-09-16T04:53:47.778758+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:47.778758+0000","last_undegraded":"2024-09-16T04:53:47.778758+0000","last_fullsized":"2024-09-16T04:53:47.778758+0000","mapping_epoch":46,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8084,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'11378","reported_seq":11476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.822641+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:47.822641+0000","last_peered":"2024-09-16T04:53:47.822641+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:47.822641+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34808425,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11377,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1968},{"shards":"2","objects":5092}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550571+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:40.550571+0000","last_peered":"2024-09-16T04:51:40.550571+0000","last_clean":"2024-09-16T04:51:40.550571+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:40.550571+0000","last_undegraded":"2024-09-16T04:51:40.550571+0000","last_fullsized":"2024-09-16T04:51:40.550571+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11729","reported_seq":11832,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.855932+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:47.855932+0000","last_peered":"2024-09-16T04:53:47.855932+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:47.855932+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36058939,"num_objects":8806,"num_object_clones":0,"num_object_copies":17612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11726,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2055},{"shards":"2","objects":5292}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11355","reported_seq":16892,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.644835+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:43.644835+0000","last_peered":"2024-09-16T04:53:43.644835+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:43.644835+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36220928,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":623,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8109,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2702,"num_bytes_recovered":11059200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8218},{"shards":"0,2","objects":623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11668","reported_seq":11809,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.656663+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:43.656663+0000","last_peered":"2024-09-16T04:53:43.656663+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:43.656663+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35832425,"num_objects":8749,"num_object_clones":0,"num_object_copies":17498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8437,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5225},{"shards":"0,2","objects":2070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":568410359,"num_objects":138779,"num_object_clones":0,"num_object_copies":277558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":623,"num_objects_unfound":0,"num_objects_dirty":138779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147851,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6057,"num_bytes_recovered":24768512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1218363392,"data_stored":1218282189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160973,"ondisk_log_size":160973,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735100,"kb_used_data":428524,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93636740,"statfs":{"total":96636764160,"available":95884021760,"internally_reserved":0,"allocated":438808576,"data_stored":438535521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593779,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625900,"kb_used_data":333212,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93745940,"statfs":{"total":96636764160,"available":95995842560,"internally_reserved":0,"allocated":341209088,"data_stored":340873411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":733936,"kb_used_data":431584,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93637904,"statfs":{"total":96636764160,"available":95885213696,"internally_reserved":0,"allocated":441942016,"data_stored":441673033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440397824,"data_stored":440394345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":340779008,"data_stored":340708835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437186560,"data_stored":437179009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:49.025 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:49.941 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:49.942 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:49.996 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-16T04:53:48.381565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":568410378,"num_objects":138781,"num_object_clones":0,"num_object_copies":277562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":623,"num_objects_unfound":0,"num_objects_dirty":138781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147853,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6058,"num_bytes_recovered":24768531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160976,"ondisk_log_size":160976,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2094936,"kb_used_data":1193320,"kb_used_omap":12,"kb_used_meta":901555,"kb_avail":281020584,"statfs":{"total":289910292480,"available":287765078016,"internally_reserved":0,"allocated":1221959680,"data_stored":1221081965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":923192321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,6,16,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23175168,"num_objects":-5658,"num_object_clones":0,"num_object_copies":-11316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-254,"num_objects_unfound":0,"num_objects_dirty":-5658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5658,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":1040384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002952"},"pg_stats":[{"pgid":"2.f","version":"47'11178","reported_seq":11258,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.660136+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:43.660136+0000","last_peered":"2024-09-16T04:53:43.660136+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:43.660136+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35061760,"num_objects":8560,"num_object_clones":0,"num_object_copies":17120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11178,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5219},{"shards":"0,2","objects":2031}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11429","reported_seq":11546,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:47.854117+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:47.854117+0000","last_peered":"2024-09-16T04:53:47.854117+0000","last_clean":"2024-09-16T04:53:47.854117+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:47.854117+0000","last_undegraded":"2024-09-16T04:53:47.854117+0000","last_fullsized":"2024-09-16T04:53:47.854117+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35082240,"num_objects":8565,"num_object_clones":0,"num_object_copies":17130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8398,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'11314","reported_seq":11454,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.822559+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:47.822559+0000","last_peered":"2024-09-16T04:53:47.822559+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:47.822559+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34992152,"num_objects":8544,"num_object_clones":0,"num_object_copies":17088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8053,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1936},{"shards":"2","objects":5225}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11615","reported_seq":11686,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:47.823850+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:47.823850+0000","last_peered":"2024-09-16T04:53:47.823850+0000","last_clean":"2024-09-16T04:53:47.823850+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:47.823850+0000","last_undegraded":"2024-09-16T04:53:47.823850+0000","last_fullsized":"2024-09-16T04:53:47.823850+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35753984,"num_objects":8729,"num_object_clones":0,"num_object_copies":17458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11615,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11369","reported_seq":11588,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:43.584304+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:43.584304+0000","last_peered":"2024-09-16T04:53:43.584304+0000","last_clean":"2024-09-16T04:53:43.584304+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:43.584304+0000","last_undegraded":"2024-09-16T04:53:43.584304+0000","last_fullsized":"2024-09-16T04:53:43.584304+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35942400,"num_objects":8775,"num_object_clones":0,"num_object_copies":17550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8401,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11489","reported_seq":11623,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.637429+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:43.637429+0000","last_peered":"2024-09-16T04:53:43.637429+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:43.637429+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36143104,"num_objects":8824,"num_object_clones":0,"num_object_copies":17648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8482,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5368},{"shards":"0,2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11261","reported_seq":11410,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.627406+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:43.627406+0000","last_peered":"2024-09-16T04:53:43.627406+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:43.627406+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35221504,"num_objects":8599,"num_object_clones":0,"num_object_copies":17198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7916,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5200},{"shards":"0,2","objects":2071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11290","reported_seq":11506,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:43.686570+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:43.686570+0000","last_peered":"2024-09-16T04:53:43.686570+0000","last_clean":"2024-09-16T04:53:43.686570+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:43.686570+0000","last_undegraded":"2024-09-16T04:53:43.686570+0000","last_fullsized":"2024-09-16T04:53:43.686570+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35741696,"num_objects":8726,"num_object_clones":0,"num_object_copies":17452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8360,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:35.473233+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:35.473233+0000","last_peered":"2024-09-16T04:51:35.473233+0000","last_clean":"2024-09-16T04:51:35.473233+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:35.473233+0000","last_undegraded":"2024-09-16T04:51:35.473233+0000","last_fullsized":"2024-09-16T04:51:35.473233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11472","reported_seq":11701,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.875030+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:47.875030+0000","last_peered":"2024-09-16T04:53:47.875030+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:47.875030+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35262464,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8223,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1906},{"shards":"2","objects":5288}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11496","reported_seq":11625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.693689+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:43.693689+0000","last_peered":"2024-09-16T04:53:43.693689+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:43.693689+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35725312,"num_objects":8722,"num_object_clones":0,"num_object_copies":17444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8151,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5257},{"shards":"0,2","objects":2082}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11369","reported_seq":11612,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.860224+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:47.860224+0000","last_peered":"2024-09-16T04:53:47.860224+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:47.860224+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35267794,"num_objects":8612,"num_object_clones":0,"num_object_copies":17224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11341,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1932},{"shards":"2","objects":5316}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550553+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:40.550553+0000","last_peered":"2024-09-16T04:51:40.550553+0000","last_clean":"2024-09-16T04:51:40.550553+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:40.550553+0000","last_undegraded":"2024-09-16T04:51:40.550553+0000","last_fullsized":"2024-09-16T04:51:40.550553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'11361","reported_seq":18218,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:47.778758+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:47.778758+0000","last_peered":"2024-09-16T04:53:47.778758+0000","last_clean":"2024-09-16T04:53:47.778758+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:47.778758+0000","last_undegraded":"2024-09-16T04:53:47.778758+0000","last_fullsized":"2024-09-16T04:53:47.778758+0000","mapping_epoch":46,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8084,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'11378","reported_seq":11476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.822641+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:47.822641+0000","last_peered":"2024-09-16T04:53:47.822641+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:47.822641+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34808425,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11377,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1968},{"shards":"2","objects":5092}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550571+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:40.550571+0000","last_peered":"2024-09-16T04:51:40.550571+0000","last_clean":"2024-09-16T04:51:40.550571+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:40.550571+0000","last_undegraded":"2024-09-16T04:51:40.550571+0000","last_fullsized":"2024-09-16T04:51:40.550571+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11729","reported_seq":11832,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.855932+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:47.855932+0000","last_peered":"2024-09-16T04:53:47.855932+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:47.855932+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36058939,"num_objects":8806,"num_object_clones":0,"num_object_copies":17612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11726,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2055},{"shards":"2","objects":5292}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11355","reported_seq":16892,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.644835+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:43.644835+0000","last_peered":"2024-09-16T04:53:43.644835+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:43.644835+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36220928,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":623,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8109,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2702,"num_bytes_recovered":11059200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8218},{"shards":"0,2","objects":623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11668","reported_seq":11809,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:43.656663+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:43.656663+0000","last_peered":"2024-09-16T04:53:43.656663+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:43.656663+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35832425,"num_objects":8749,"num_object_clones":0,"num_object_copies":17498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8437,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5225},{"shards":"0,2","objects":2070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":568410359,"num_objects":138779,"num_object_clones":0,"num_object_copies":277558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":623,"num_objects_unfound":0,"num_objects_dirty":138779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":147851,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6057,"num_bytes_recovered":24768512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1218363392,"data_stored":1218282189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160973,"ondisk_log_size":160973,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735100,"kb_used_data":428524,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93636740,"statfs":{"total":96636764160,"available":95884021760,"internally_reserved":0,"allocated":438808576,"data_stored":438535521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593779,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625900,"kb_used_data":333212,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93745940,"statfs":{"total":96636764160,"available":95995842560,"internally_reserved":0,"allocated":341209088,"data_stored":340873411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":733936,"kb_used_data":431584,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93637904,"statfs":{"total":96636764160,"available":95885213696,"internally_reserved":0,"allocated":441942016,"data_stored":441673033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440397824,"data_stored":440394345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":340779008,"data_stored":340708835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437186560,"data_stored":437179009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:49.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:53:49.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'11178", 'reported_seq': 11258, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:43.660136+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:53:43.660136+0000', 'last_peered': '2024-09-16T04:53:43.660136+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:53:43.660136+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35061760, 'num_objects': 8560, 'num_object_clones': 0, 'num_object_copies': 17120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8560, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11178, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5219}, {'shards': '0,2', 'objects': 2031}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:49.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:53:49.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'11314", 'reported_seq': 11454, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:47.822559+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:53:47.822559+0000', 'last_peered': '2024-09-16T04:53:47.822559+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:53:47.822559+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34992152, 'num_objects': 8544, 'num_object_clones': 0, 'num_object_copies': 17088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8544, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8053, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1936}, {'shards': '2', 'objects': 5225}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:49.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:53:49.998 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'11489", 'reported_seq': 11623, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:43.637429+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:53:43.637429+0000', 'last_peered': '2024-09-16T04:53:43.637429+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:53:43.637429+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36143104, 'num_objects': 8824, 'num_object_clones': 0, 'num_object_copies': 17648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8824, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8482, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5368}, {'shards': '0,2', 'objects': 2126}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:49.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:53:49.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'11261", 'reported_seq': 11410, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:43.627406+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:53:43.627406+0000', 'last_peered': '2024-09-16T04:53:43.627406+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:53:43.627406+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35221504, 'num_objects': 8599, 'num_object_clones': 0, 'num_object_copies': 17198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8599, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7916, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5200}, {'shards': '0,2', 'objects': 2071}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:49.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:53:49.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'11472", 'reported_seq': 11701, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:47.875030+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:53:47.875030+0000', 'last_peered': '2024-09-16T04:53:47.875030+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:53:47.875030+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35262464, 'num_objects': 8609, 'num_object_clones': 0, 'num_object_copies': 17218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8609, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8223, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1906}, {'shards': '2', 'objects': 5288}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:49.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:53:49.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'11496", 'reported_seq': 11625, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:43.693689+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:53:43.693689+0000', 'last_peered': '2024-09-16T04:53:43.693689+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:53:43.693689+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35725312, 'num_objects': 8722, 'num_object_clones': 0, 'num_object_copies': 17444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8722, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8151, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5257}, {'shards': '0,2', 'objects': 2082}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:49.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:53:49.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'11369", 'reported_seq': 11612, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:47.860224+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:53:47.860224+0000', 'last_peered': '2024-09-16T04:53:47.860224+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:53:47.860224+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35267794, 'num_objects': 8612, 'num_object_clones': 0, 'num_object_copies': 17224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8612, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11341, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1932}, {'shards': '2', 'objects': 5316}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:49.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:53:50.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'11378", 'reported_seq': 11476, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:47.822641+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:53:47.822641+0000', 'last_peered': '2024-09-16T04:53:47.822641+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:53:47.822641+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34808425, 'num_objects': 8499, 'num_object_clones': 0, 'num_object_copies': 16998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8499, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11377, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1968}, {'shards': '2', 'objects': 5092}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:50.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:53:50.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'11729", 'reported_seq': 11832, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:47.855932+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:53:47.855932+0000', 'last_peered': '2024-09-16T04:53:47.855932+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:53:47.855932+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36058939, 'num_objects': 8806, 'num_object_clones': 0, 'num_object_copies': 17612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8806, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11726, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2055}, {'shards': '2', 'objects': 5292}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:50.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:53:50.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'11355", 'reported_seq': 16892, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:43.644835+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:53:43.644835+0000', 'last_peered': '2024-09-16T04:53:43.644835+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:53:43.644835+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36220928, 'num_objects': 8843, 'num_object_clones': 0, 'num_object_copies': 17686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 623, 'num_objects_unfound': 0, 'num_objects_dirty': 8843, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8109, 'num_write_kb': 27420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2702, 'num_bytes_recovered': 11059200, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8218}, {'shards': '0,2', 'objects': 623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:50.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:53:50.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'11668", 'reported_seq': 11809, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:43.656663+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:53:43.656663+0000', 'last_peered': '2024-09-16T04:53:43.656663+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:53:43.656663+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35832425, 'num_objects': 8749, 'num_object_clones': 0, 'num_object_copies': 17498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8749, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8437, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5225}, {'shards': '0,2', 'objects': 2070}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:50.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:53:51.021 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:51.022 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/275182 objects degraded (12.378%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":378,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":137591,"data_bytes":563536138,"bytes_used":2179219456,"bytes_avail":287731073024,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":275182,"degraded_ratio":0.12377989839451707,"misplaced_objects":484,"misplaced_total":275182,"misplaced_ratio":0.0017588359703759693,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":134110,"recovering_keys_per_sec":0,"num_objects_recovered":393,"num_bytes_recovered":1609728,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":473},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:53:51.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:53:51.077 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:51.948 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:51.949 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:52.039 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-16T04:53:50.382056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":563536138,"num_objects":137591,"num_object_clones":0,"num_object_copies":275182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":484,"num_objects_unfound":0,"num_objects_dirty":137591,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149043,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6197,"num_bytes_recovered":25337875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160566,"ondisk_log_size":160566,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2128144,"kb_used_data":1186272,"kb_used_omap":12,"kb_used_meta":941811,"kb_avail":280987376,"statfs":{"total":289910292480,"available":287731073024,"internally_reserved":0,"allocated":1214742528,"data_stored":1213864813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":964414465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,6,6,14,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23269376,"num_objects":-5681,"num_object_clones":0,"num_object_copies":-11362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-393,"num_objects_unfound":0,"num_objects_dirty":-5681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5681,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":1609728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003014"},"pg_stats":[{"pgid":"2.f","version":"47'11324","reported_seq":11402,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:48.743384+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:48.743384+0000","last_peered":"2024-09-16T04:53:48.743384+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:48.743384+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34463744,"num_objects":8414,"num_object_clones":0,"num_object_copies":16828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11324,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5073},{"shards":"0,2","objects":1885}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11429","reported_seq":11546,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:47.854117+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:47.854117+0000","last_peered":"2024-09-16T04:53:47.854117+0000","last_clean":"2024-09-16T04:53:47.854117+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:47.854117+0000","last_undegraded":"2024-09-16T04:53:47.854117+0000","last_fullsized":"2024-09-16T04:53:47.854117+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35082240,"num_objects":8565,"num_object_clones":0,"num_object_copies":17130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8398,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'11314","reported_seq":11454,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.822559+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:47.822559+0000","last_peered":"2024-09-16T04:53:47.822559+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:47.822559+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34992152,"num_objects":8544,"num_object_clones":0,"num_object_copies":17088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8053,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1936},{"shards":"2","objects":5225}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11615","reported_seq":11686,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:47.823850+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:47.823850+0000","last_peered":"2024-09-16T04:53:47.823850+0000","last_clean":"2024-09-16T04:53:47.823850+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:47.823850+0000","last_undegraded":"2024-09-16T04:53:47.823850+0000","last_fullsized":"2024-09-16T04:53:47.823850+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35753984,"num_objects":8729,"num_object_clones":0,"num_object_copies":17458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11615,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11506","reported_seq":11725,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:48.739550+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:48.739550+0000","last_peered":"2024-09-16T04:53:48.739550+0000","last_clean":"2024-09-16T04:53:48.739550+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:48.739550+0000","last_undegraded":"2024-09-16T04:53:48.739550+0000","last_fullsized":"2024-09-16T04:53:48.739550+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35381248,"num_objects":8638,"num_object_clones":0,"num_object_copies":17276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8538,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11640","reported_seq":11774,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:48.720887+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:48.720887+0000","last_peered":"2024-09-16T04:53:48.720887+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:48.720887+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35524608,"num_objects":8673,"num_object_clones":0,"num_object_copies":17346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8633,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5217},{"shards":"0,2","objects":1975}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11415","reported_seq":11564,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:48.662929+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:48.662929+0000","last_peered":"2024-09-16T04:53:48.662929+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:48.662929+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34590720,"num_objects":8445,"num_object_clones":0,"num_object_copies":16890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8070,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5046},{"shards":"0,2","objects":1917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11434","reported_seq":11651,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:48.698324+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:48.698324+0000","last_peered":"2024-09-16T04:53:48.698324+0000","last_clean":"2024-09-16T04:53:48.698324+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:48.698324+0000","last_undegraded":"2024-09-16T04:53:48.698324+0000","last_fullsized":"2024-09-16T04:53:48.698324+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35151872,"num_objects":8582,"num_object_clones":0,"num_object_copies":17164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8504,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:35.473233+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:35.473233+0000","last_peered":"2024-09-16T04:51:35.473233+0000","last_clean":"2024-09-16T04:51:35.473233+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:35.473233+0000","last_undegraded":"2024-09-16T04:51:35.473233+0000","last_fullsized":"2024-09-16T04:51:35.473233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11472","reported_seq":11701,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.875030+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:47.875030+0000","last_peered":"2024-09-16T04:53:47.875030+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:47.875030+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35262464,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8223,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1906},{"shards":"2","objects":5288}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11653","reported_seq":11783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:48.720931+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:48.720931+0000","last_peered":"2024-09-16T04:53:48.720931+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:48.720931+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35082240,"num_objects":8565,"num_object_clones":0,"num_object_copies":17130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8308,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5100},{"shards":"0,2","objects":1925}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11369","reported_seq":11612,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.860224+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:47.860224+0000","last_peered":"2024-09-16T04:53:47.860224+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:47.860224+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35267794,"num_objects":8612,"num_object_clones":0,"num_object_copies":17224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11341,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1932},{"shards":"2","objects":5316}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550553+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:40.550553+0000","last_peered":"2024-09-16T04:51:40.550553+0000","last_clean":"2024-09-16T04:51:40.550553+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:40.550553+0000","last_undegraded":"2024-09-16T04:51:40.550553+0000","last_fullsized":"2024-09-16T04:51:40.550553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'11361","reported_seq":18218,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:47.778758+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:47.778758+0000","last_peered":"2024-09-16T04:53:47.778758+0000","last_clean":"2024-09-16T04:53:47.778758+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:47.778758+0000","last_undegraded":"2024-09-16T04:53:47.778758+0000","last_fullsized":"2024-09-16T04:53:47.778758+0000","mapping_epoch":46,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8084,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'11378","reported_seq":11476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.822641+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:47.822641+0000","last_peered":"2024-09-16T04:53:47.822641+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:47.822641+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34808425,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11377,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1968},{"shards":"2","objects":5092}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550571+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:40.550571+0000","last_peered":"2024-09-16T04:51:40.550571+0000","last_clean":"2024-09-16T04:51:40.550571+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:40.550571+0000","last_undegraded":"2024-09-16T04:51:40.550571+0000","last_fullsized":"2024-09-16T04:51:40.550571+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11729","reported_seq":11832,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:47.855932+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:47.855932+0000","last_peered":"2024-09-16T04:53:47.855932+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:47.855932+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36058939,"num_objects":8806,"num_object_clones":0,"num_object_copies":17612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11726,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2055},{"shards":"2","objects":5292}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11502","reported_seq":17317,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:48.749616+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:48.749616+0000","last_peered":"2024-09-16T04:53:48.749616+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:48.749616+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35618816,"num_objects":8696,"num_object_clones":0,"num_object_copies":17392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":484,"num_objects_unfound":0,"num_objects_dirty":8696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8256,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2841,"num_bytes_recovered":11628544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8210},{"shards":"0,2","objects":484}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.150675+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:33.150675+0000","last_peered":"2024-09-16T04:51:33.150675+0000","last_clean":"2024-09-16T04:51:33.150675+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:33.150675+0000","last_undegraded":"2024-09-16T04:51:33.150675+0000","last_fullsized":"2024-09-16T04:51:33.150675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11822","reported_seq":11961,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:48.748247+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:48.748247+0000","last_peered":"2024-09-16T04:53:48.748247+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:48.748247+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35201641,"num_objects":8595,"num_object_clones":0,"num_object_copies":17190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8591,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5071},{"shards":"0,2","objects":1916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":563536119,"num_objects":137589,"num_object_clones":0,"num_object_copies":275178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":484,"num_objects_unfound":0,"num_objects_dirty":137589,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149041,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6196,"num_bytes_recovered":25337856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1212227584,"data_stored":1212146381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160563,"ondisk_log_size":160563,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735100,"kb_used_data":428524,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93636740,"statfs":{"total":96636764160,"available":95884021760,"internally_reserved":0,"allocated":438808576,"data_stored":438535521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593779,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625900,"kb_used_data":333212,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93745940,"statfs":{"total":96636764160,"available":95995842560,"internally_reserved":0,"allocated":341209088,"data_stored":340873411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":767144,"kb_used_data":424536,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93604696,"statfs":{"total":96636764160,"available":95851208704,"internally_reserved":0,"allocated":434724864,"data_stored":434455881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,3,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":434262016,"data_stored":434258537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":340779008,"data_stored":340708835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437186560,"data_stored":437179009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:55.041 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:55.950 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:55.963 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:56.019 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-16T04:53:54.383037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":554238218,"num_objects":135321,"num_object_clones":0,"num_object_copies":270642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":337,"num_objects_unfound":0,"num_objects_dirty":135321,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":151313,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6344,"num_bytes_recovered":25939987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2134324,"kb_used_data":1174396,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":280981196,"statfs":{"total":289910292480,"available":287724744704,"internally_reserved":0,"allocated":1202581504,"data_stored":1201703789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":982895617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,8,6,10,1,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27975680,"num_objects":-6830,"num_object_clones":0,"num_object_copies":-13660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-431,"num_objects_unfound":0,"num_objects_dirty":-6830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6830,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":431,"num_bytes_recovered":1765376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003024"},"pg_stats":[{"pgid":"2.f","version":"47'11498","reported_seq":11576,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.723412+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:53.723412+0000","last_peered":"2024-09-16T04:53:53.723412+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:53.723412+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33751040,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11498,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4899},{"shards":"0,2","objects":1711}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11593","reported_seq":11710,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:52.869661+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:52.869661+0000","last_peered":"2024-09-16T04:53:52.869661+0000","last_clean":"2024-09-16T04:53:52.869661+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:52.869661+0000","last_undegraded":"2024-09-16T04:53:52.869661+0000","last_fullsized":"2024-09-16T04:53:52.869661+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34410496,"num_objects":8401,"num_object_clones":0,"num_object_copies":16802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8562,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'11450","reported_seq":11590,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:52.863078+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:52.863078+0000","last_peered":"2024-09-16T04:53:52.863078+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:52.863078+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34435096,"num_objects":8408,"num_object_clones":0,"num_object_copies":16816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8408,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8189,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1800},{"shards":"2","objects":5089}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11762","reported_seq":11833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:52.851350+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:52.851350+0000","last_peered":"2024-09-16T04:53:52.851350+0000","last_clean":"2024-09-16T04:53:52.851350+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:52.851350+0000","last_undegraded":"2024-09-16T04:53:52.851350+0000","last_fullsized":"2024-09-16T04:53:52.851350+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35151872,"num_objects":8582,"num_object_clones":0,"num_object_copies":17164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11762,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11640","reported_seq":11858,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:53.775695+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:53.775695+0000","last_peered":"2024-09-16T04:53:53.775695+0000","last_clean":"2024-09-16T04:53:53.775695+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:53.775695+0000","last_undegraded":"2024-09-16T04:53:53.775695+0000","last_fullsized":"2024-09-16T04:53:53.775695+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34832384,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8672,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11772","reported_seq":11906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.646450+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:53.646450+0000","last_peered":"2024-09-16T04:53:53.646450+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:53.646450+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8765,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5085},{"shards":"0,2","objects":1843}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11569","reported_seq":11718,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.712497+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:53.712497+0000","last_peered":"2024-09-16T04:53:53.712497+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:53.712497+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33959936,"num_objects":8291,"num_object_clones":0,"num_object_copies":16582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8224,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4892},{"shards":"0,2","objects":1763}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11574","reported_seq":11791,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:53.738646+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:53.738646+0000","last_peered":"2024-09-16T04:53:53.738646+0000","last_clean":"2024-09-16T04:53:53.738646+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:53.738646+0000","last_undegraded":"2024-09-16T04:53:53.738646+0000","last_fullsized":"2024-09-16T04:53:53.738646+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8644,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:35.473233+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:35.473233+0000","last_peered":"2024-09-16T04:51:35.473233+0000","last_clean":"2024-09-16T04:51:35.473233+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:35.473233+0000","last_undegraded":"2024-09-16T04:51:35.473233+0000","last_fullsized":"2024-09-16T04:51:35.473233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11602","reported_seq":11831,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:52.914606+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:52.914606+0000","last_peered":"2024-09-16T04:53:52.914606+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:52.914606+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34729984,"num_objects":8479,"num_object_clones":0,"num_object_copies":16958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8353,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1776},{"shards":"2","objects":5158}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11788","reported_seq":11919,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.801234+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:53.801234+0000","last_peered":"2024-09-16T04:53:53.801234+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:53.801234+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34529280,"num_objects":8430,"num_object_clones":0,"num_object_copies":16860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8443,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4965},{"shards":"0,2","objects":1790}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11508","reported_seq":11750,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:52.894123+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:52.894123+0000","last_peered":"2024-09-16T04:53:52.894123+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:52.894123+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34698450,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11480,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1793},{"shards":"2","objects":5177}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550553+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:40.550553+0000","last_peered":"2024-09-16T04:51:40.550553+0000","last_clean":"2024-09-16T04:51:40.550553+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:40.550553+0000","last_undegraded":"2024-09-16T04:51:40.550553+0000","last_fullsized":"2024-09-16T04:51:40.550553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'11511","reported_seq":18368,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:52.839337+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:52.839337+0000","last_peered":"2024-09-16T04:53:52.839337+0000","last_clean":"2024-09-16T04:53:52.839337+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:52.839337+0000","last_undegraded":"2024-09-16T04:53:52.839337+0000","last_fullsized":"2024-09-16T04:53:52.839337+0000","mapping_epoch":46,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34680832,"num_objects":8467,"num_object_clones":0,"num_object_copies":16934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8234,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'11506","reported_seq":11604,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:52.831149+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:52.831149+0000","last_peered":"2024-09-16T04:53:52.831149+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:52.831149+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34284137,"num_objects":8371,"num_object_clones":0,"num_object_copies":16742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11505,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1840},{"shards":"2","objects":4964}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550571+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:40.550571+0000","last_peered":"2024-09-16T04:51:40.550571+0000","last_clean":"2024-09-16T04:51:40.550571+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:40.550571+0000","last_undegraded":"2024-09-16T04:51:40.550571+0000","last_fullsized":"2024-09-16T04:51:40.550571+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11872","reported_seq":11975,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:52.867669+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:52.867669+0000","last_peered":"2024-09-16T04:53:52.867669+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:52.867669+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35473211,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11869,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1912},{"shards":"2","objects":5149}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11637","reported_seq":17746,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.796953+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:53.796953+0000","last_peered":"2024-09-16T04:53:53.796953+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:53.796953+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35065856,"num_objects":8561,"num_object_clones":0,"num_object_copies":17122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":337,"num_objects_unfound":0,"num_objects_dirty":8561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8391,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2988,"num_bytes_recovered":12230656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8222},{"shards":"0,2","objects":337}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:45.615397+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:45.615397+0000","last_peered":"2024-09-16T04:51:45.615397+0000","last_clean":"2024-09-16T04:51:45.615397+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:45.615397+0000","last_undegraded":"2024-09-16T04:51:45.615397+0000","last_fullsized":"2024-09-16T04:51:45.615397+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11951","reported_seq":12088,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.739016+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:53.739016+0000","last_peered":"2024-09-16T04:53:53.739016+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:53.739016+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34673257,"num_objects":8466,"num_object_clones":0,"num_object_copies":16932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8720,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4942},{"shards":"0,2","objects":1787}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":554238199,"num_objects":135319,"num_object_clones":0,"num_object_copies":270638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":337,"num_objects_unfound":0,"num_objects_dirty":135319,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":151311,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6343,"num_bytes_recovered":25939968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1200168960,"data_stored":1200087757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751600,"kb_used_data":422812,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93620240,"statfs":{"total":96636764160,"available":95867125760,"internally_reserved":0,"allocated":432959488,"data_stored":432686433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":336653993},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593780,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617140,"kb_used_data":332772,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93754700,"statfs":{"total":96636764160,"available":96004812800,"internally_reserved":0,"allocated":340758528,"data_stored":340422851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":765584,"kb_used_data":418812,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93606256,"statfs":{"total":96636764160,"available":95852806144,"internally_reserved":0,"allocated":428863488,"data_stored":428594505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,3,1,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":428437504,"data_stored":428434025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":340299776,"data_stored":340229603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431431680,"data_stored":431424129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:56.021 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:56.858 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:56.912 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-16T04:53:56.383518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":554238218,"num_objects":135321,"num_object_clones":0,"num_object_copies":270642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":337,"num_objects_unfound":0,"num_objects_dirty":135321,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":151313,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6344,"num_bytes_recovered":25939987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2134324,"kb_used_data":1174396,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":280981196,"statfs":{"total":289910292480,"available":287724744704,"internally_reserved":0,"allocated":1202581504,"data_stored":1201703789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":982895617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,8,6,10,1,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18653184,"num_objects":-4554,"num_object_clones":0,"num_object_copies":-9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-286,"num_objects_unfound":0,"num_objects_dirty":-4554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4554,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":1171456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003044"},"pg_stats":[{"pgid":"2.f","version":"47'11498","reported_seq":11576,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.723412+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:53.723412+0000","last_peered":"2024-09-16T04:53:53.723412+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:53.723412+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33751040,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11498,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4899},{"shards":"0,2","objects":1711}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11593","reported_seq":11710,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:52.869661+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:52.869661+0000","last_peered":"2024-09-16T04:53:52.869661+0000","last_clean":"2024-09-16T04:53:52.869661+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:52.869661+0000","last_undegraded":"2024-09-16T04:53:52.869661+0000","last_fullsized":"2024-09-16T04:53:52.869661+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34410496,"num_objects":8401,"num_object_clones":0,"num_object_copies":16802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8562,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'11450","reported_seq":11590,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:52.863078+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:52.863078+0000","last_peered":"2024-09-16T04:53:52.863078+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:52.863078+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34435096,"num_objects":8408,"num_object_clones":0,"num_object_copies":16816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8408,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8189,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1800},{"shards":"2","objects":5089}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11762","reported_seq":11833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:52.851350+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:52.851350+0000","last_peered":"2024-09-16T04:53:52.851350+0000","last_clean":"2024-09-16T04:53:52.851350+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:52.851350+0000","last_undegraded":"2024-09-16T04:53:52.851350+0000","last_fullsized":"2024-09-16T04:53:52.851350+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35151872,"num_objects":8582,"num_object_clones":0,"num_object_copies":17164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11762,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11640","reported_seq":11858,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:53.775695+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:53.775695+0000","last_peered":"2024-09-16T04:53:53.775695+0000","last_clean":"2024-09-16T04:53:53.775695+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:53.775695+0000","last_undegraded":"2024-09-16T04:53:53.775695+0000","last_fullsized":"2024-09-16T04:53:53.775695+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34832384,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8672,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11772","reported_seq":11906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.646450+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:53.646450+0000","last_peered":"2024-09-16T04:53:53.646450+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:53.646450+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8765,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5085},{"shards":"0,2","objects":1843}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11569","reported_seq":11718,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.712497+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:53.712497+0000","last_peered":"2024-09-16T04:53:53.712497+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:53.712497+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33959936,"num_objects":8291,"num_object_clones":0,"num_object_copies":16582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8224,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4892},{"shards":"0,2","objects":1763}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11574","reported_seq":11791,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:53.738646+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:53.738646+0000","last_peered":"2024-09-16T04:53:53.738646+0000","last_clean":"2024-09-16T04:53:53.738646+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:53.738646+0000","last_undegraded":"2024-09-16T04:53:53.738646+0000","last_fullsized":"2024-09-16T04:53:53.738646+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8644,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:35.473233+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:35.473233+0000","last_peered":"2024-09-16T04:51:35.473233+0000","last_clean":"2024-09-16T04:51:35.473233+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:35.473233+0000","last_undegraded":"2024-09-16T04:51:35.473233+0000","last_fullsized":"2024-09-16T04:51:35.473233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11602","reported_seq":11831,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:52.914606+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:52.914606+0000","last_peered":"2024-09-16T04:53:52.914606+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:52.914606+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34729984,"num_objects":8479,"num_object_clones":0,"num_object_copies":16958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8353,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1776},{"shards":"2","objects":5158}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11788","reported_seq":11919,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.801234+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:53.801234+0000","last_peered":"2024-09-16T04:53:53.801234+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:53.801234+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34529280,"num_objects":8430,"num_object_clones":0,"num_object_copies":16860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8443,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4965},{"shards":"0,2","objects":1790}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11508","reported_seq":11750,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:52.894123+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:52.894123+0000","last_peered":"2024-09-16T04:53:52.894123+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:52.894123+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34698450,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11480,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1793},{"shards":"2","objects":5177}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550553+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:40.550553+0000","last_peered":"2024-09-16T04:51:40.550553+0000","last_clean":"2024-09-16T04:51:40.550553+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:40.550553+0000","last_undegraded":"2024-09-16T04:51:40.550553+0000","last_fullsized":"2024-09-16T04:51:40.550553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'11511","reported_seq":18368,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:52.839337+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:52.839337+0000","last_peered":"2024-09-16T04:53:52.839337+0000","last_clean":"2024-09-16T04:53:52.839337+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:52.839337+0000","last_undegraded":"2024-09-16T04:53:52.839337+0000","last_fullsized":"2024-09-16T04:53:52.839337+0000","mapping_epoch":46,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34680832,"num_objects":8467,"num_object_clones":0,"num_object_copies":16934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8234,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'11506","reported_seq":11604,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:52.831149+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:52.831149+0000","last_peered":"2024-09-16T04:53:52.831149+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:52.831149+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34284137,"num_objects":8371,"num_object_clones":0,"num_object_copies":16742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11505,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1840},{"shards":"2","objects":4964}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550571+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:40.550571+0000","last_peered":"2024-09-16T04:51:40.550571+0000","last_clean":"2024-09-16T04:51:40.550571+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:40.550571+0000","last_undegraded":"2024-09-16T04:51:40.550571+0000","last_fullsized":"2024-09-16T04:51:40.550571+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'11872","reported_seq":11975,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:52.867669+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:52.867669+0000","last_peered":"2024-09-16T04:53:52.867669+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:52.867669+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35473211,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11869,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1912},{"shards":"2","objects":5149}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11637","reported_seq":17746,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.796953+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:53.796953+0000","last_peered":"2024-09-16T04:53:53.796953+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:53.796953+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35065856,"num_objects":8561,"num_object_clones":0,"num_object_copies":17122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":337,"num_objects_unfound":0,"num_objects_dirty":8561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8391,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2988,"num_bytes_recovered":12230656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8222},{"shards":"0,2","objects":337}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:45.615397+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:45.615397+0000","last_peered":"2024-09-16T04:51:45.615397+0000","last_clean":"2024-09-16T04:51:45.615397+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:45.615397+0000","last_undegraded":"2024-09-16T04:51:45.615397+0000","last_fullsized":"2024-09-16T04:51:45.615397+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.155476+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:33.155476+0000","last_peered":"2024-09-16T04:51:33.155476+0000","last_clean":"2024-09-16T04:51:33.155476+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:33.155476+0000","last_undegraded":"2024-09-16T04:51:33.155476+0000","last_fullsized":"2024-09-16T04:51:33.155476+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11951","reported_seq":12088,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.739016+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:53.739016+0000","last_peered":"2024-09-16T04:53:53.739016+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:53.739016+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34673257,"num_objects":8466,"num_object_clones":0,"num_object_copies":16932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8720,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4942},{"shards":"0,2","objects":1787}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":554238199,"num_objects":135319,"num_object_clones":0,"num_object_copies":270638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":337,"num_objects_unfound":0,"num_objects_dirty":135319,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":151311,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6343,"num_bytes_recovered":25939968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1200168960,"data_stored":1200087757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751600,"kb_used_data":422812,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93620240,"statfs":{"total":96636764160,"available":95867125760,"internally_reserved":0,"allocated":432959488,"data_stored":432686433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":336653993},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593780,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617140,"kb_used_data":332772,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93754700,"statfs":{"total":96636764160,"available":96004812800,"internally_reserved":0,"allocated":340758528,"data_stored":340422851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":765584,"kb_used_data":418812,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93606256,"statfs":{"total":96636764160,"available":95852806144,"internally_reserved":0,"allocated":428863488,"data_stored":428594505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,3,1,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":428437504,"data_stored":428434025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":340299776,"data_stored":340229603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431431680,"data_stored":431424129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:53:56.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:53:56.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'11498", 'reported_seq': 11576, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:53.723412+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:53:53.723412+0000', 'last_peered': '2024-09-16T04:53:53.723412+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:53:53.723412+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33751040, 'num_objects': 8240, 'num_object_clones': 0, 'num_object_copies': 16480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8240, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11498, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4899}, {'shards': '0,2', 'objects': 1711}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:56.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:53:56.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'11450", 'reported_seq': 11590, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:52.863078+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:53:52.863078+0000', 'last_peered': '2024-09-16T04:53:52.863078+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:53:52.863078+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34435096, 'num_objects': 8408, 'num_object_clones': 0, 'num_object_copies': 16816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8408, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8189, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1800}, {'shards': '2', 'objects': 5089}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:56.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:53:56.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'11772", 'reported_seq': 11906, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:53.646450+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:53:53.646450+0000', 'last_peered': '2024-09-16T04:53:53.646450+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:53:53.646450+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34983936, 'num_objects': 8541, 'num_object_clones': 0, 'num_object_copies': 17082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8541, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8765, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5085}, {'shards': '0,2', 'objects': 1843}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:56.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:53:56.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'11569", 'reported_seq': 11718, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:53.712497+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:53:53.712497+0000', 'last_peered': '2024-09-16T04:53:53.712497+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:53:53.712497+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33959936, 'num_objects': 8291, 'num_object_clones': 0, 'num_object_copies': 16582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8291, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8224, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4892}, {'shards': '0,2', 'objects': 1763}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:56.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:53:56.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'11602", 'reported_seq': 11831, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:52.914606+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:53:52.914606+0000', 'last_peered': '2024-09-16T04:53:52.914606+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:53:52.914606+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34729984, 'num_objects': 8479, 'num_object_clones': 0, 'num_object_copies': 16958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8479, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8353, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1776}, {'shards': '2', 'objects': 5158}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:56.914 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:53:56.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'11788", 'reported_seq': 11919, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:53.801234+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:53:53.801234+0000', 'last_peered': '2024-09-16T04:53:53.801234+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:53:53.801234+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34529280, 'num_objects': 8430, 'num_object_clones': 0, 'num_object_copies': 16860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8430, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8443, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4965}, {'shards': '0,2', 'objects': 1790}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:56.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:53:56.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'11508", 'reported_seq': 11750, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:52.894123+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:53:52.894123+0000', 'last_peered': '2024-09-16T04:53:52.894123+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:53:52.894123+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34698450, 'num_objects': 8473, 'num_object_clones': 0, 'num_object_copies': 16946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11480, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1793}, {'shards': '2', 'objects': 5177}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:56.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:53:56.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'11506", 'reported_seq': 11604, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:52.831149+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:53:52.831149+0000', 'last_peered': '2024-09-16T04:53:52.831149+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:53:52.831149+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34284137, 'num_objects': 8371, 'num_object_clones': 0, 'num_object_copies': 16742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8371, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11505, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1840}, {'shards': '2', 'objects': 4964}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:56.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:53:56.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'11872", 'reported_seq': 11975, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:52.867669+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:53:52.867669+0000', 'last_peered': '2024-09-16T04:53:52.867669+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:53:52.867669+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35473211, 'num_objects': 8663, 'num_object_clones': 0, 'num_object_copies': 17326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8663, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11869, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1912}, {'shards': '2', 'objects': 5149}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:53:56.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:53:56.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'11637", 'reported_seq': 17746, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:53.796953+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:53:53.796953+0000', 'last_peered': '2024-09-16T04:53:53.796953+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:53:53.796953+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35065856, 'num_objects': 8561, 'num_object_clones': 0, 'num_object_copies': 17122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 337, 'num_objects_unfound': 0, 'num_objects_dirty': 8561, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8391, 'num_write_kb': 27420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2988, 'num_bytes_recovered': 12230656, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8222}, {'shards': '0,2', 'objects': 337}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:56.915 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:53:56.916 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'11951", 'reported_seq': 12088, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:53.739016+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:53:53.739016+0000', 'last_peered': '2024-09-16T04:53:53.739016+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:53:53.739016+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34673257, 'num_objects': 8466, 'num_object_clones': 0, 'num_object_copies': 16932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8466, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8720, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4942}, {'shards': '0,2', 'objects': 1787}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:53:56.916 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:53:57.820 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:57.820 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/270642 objects degraded (12.586%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":385,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":135321,"data_bytes":554238218,"bytes_used":2185547776,"bytes_avail":287724744704,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":270642,"degraded_ratio":0.12585629724876404,"misplaced_objects":337,"misplaced_total":270642,"misplaced_ratio":0.0012451873692922754,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":97596,"recovering_keys_per_sec":0,"num_objects_recovered":286,"num_bytes_recovered":1171456,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":379},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:53:57.868 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:53:57.869 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:53:58.593 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:53:58.595 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:53:58.639 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-16T04:53:58.383974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":549445898,"num_objects":134151,"num_object_clones":0,"num_object_copies":268302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":337,"num_objects_unfound":0,"num_objects_dirty":134151,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152483,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6344,"num_bytes_recovered":25939987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160906,"ondisk_log_size":160906,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2140988,"kb_used_data":1167176,"kb_used_omap":12,"kb_used_meta":973747,"kb_avail":280974532,"statfs":{"total":289910292480,"available":287717920768,"internally_reserved":0,"allocated":1195188224,"data_stored":1194310509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":997116929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,1,6,8,4,1,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23445504,"num_objects":-5724,"num_object_clones":0,"num_object_copies":-11448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-286,"num_objects_unfound":0,"num_objects_dirty":-5724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5724,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":1171456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002910"},"pg_stats":[{"pgid":"2.f","version":"47'11498","reported_seq":11576,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.723412+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:53.723412+0000","last_peered":"2024-09-16T04:53:53.723412+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:53.723412+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33751040,"num_objects":8240,"num_object_clones":0,"num_object_copies":16480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11498,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4899},{"shards":"0,2","objects":1711}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11732","reported_seq":11849,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:57.881400+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:57.881400+0000","last_peered":"2024-09-16T04:53:57.881400+0000","last_clean":"2024-09-16T04:53:57.881400+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:57.881400+0000","last_undegraded":"2024-09-16T04:53:57.881400+0000","last_fullsized":"2024-09-16T04:53:57.881400+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33841152,"num_objects":8262,"num_object_clones":0,"num_object_copies":16524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8701,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'11606","reported_seq":11746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.891770+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:57.891770+0000","last_peered":"2024-09-16T04:53:57.891770+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:57.891770+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33796120,"num_objects":8252,"num_object_clones":0,"num_object_copies":16504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8252,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8345,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1644},{"shards":"2","objects":4933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11898","reported_seq":11969,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:57.903824+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:57.903824+0000","last_peered":"2024-09-16T04:53:57.903824+0000","last_clean":"2024-09-16T04:53:57.903824+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:57.903824+0000","last_undegraded":"2024-09-16T04:53:57.903824+0000","last_fullsized":"2024-09-16T04:53:57.903824+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34594816,"num_objects":8446,"num_object_clones":0,"num_object_copies":16892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11898,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11640","reported_seq":11858,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:53.775695+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:53.775695+0000","last_peered":"2024-09-16T04:53:53.775695+0000","last_clean":"2024-09-16T04:53:53.775695+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:53.775695+0000","last_undegraded":"2024-09-16T04:53:53.775695+0000","last_fullsized":"2024-09-16T04:53:53.775695+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34832384,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8672,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11772","reported_seq":11906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.646450+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:53.646450+0000","last_peered":"2024-09-16T04:53:53.646450+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:53.646450+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8765,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5085},{"shards":"0,2","objects":1843}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11569","reported_seq":11718,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.712497+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:53.712497+0000","last_peered":"2024-09-16T04:53:53.712497+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:53.712497+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33959936,"num_objects":8291,"num_object_clones":0,"num_object_copies":16582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8224,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4892},{"shards":"0,2","objects":1763}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11574","reported_seq":11791,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:53.738646+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:53.738646+0000","last_peered":"2024-09-16T04:53:53.738646+0000","last_clean":"2024-09-16T04:53:53.738646+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:53.738646+0000","last_undegraded":"2024-09-16T04:53:53.738646+0000","last_fullsized":"2024-09-16T04:53:53.738646+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8644,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:35.473233+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:35.473233+0000","last_peered":"2024-09-16T04:51:35.473233+0000","last_clean":"2024-09-16T04:51:35.473233+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:35.473233+0000","last_undegraded":"2024-09-16T04:51:35.473233+0000","last_fullsized":"2024-09-16T04:51:35.473233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11735","reported_seq":11963,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.951367+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:57.951367+0000","last_peered":"2024-09-16T04:53:57.951367+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:57.951367+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34185216,"num_objects":8346,"num_object_clones":0,"num_object_copies":16692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8486,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1643},{"shards":"2","objects":5025}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11788","reported_seq":11919,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.801234+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:53.801234+0000","last_peered":"2024-09-16T04:53:53.801234+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:53.801234+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34529280,"num_objects":8430,"num_object_clones":0,"num_object_copies":16860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8443,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4965},{"shards":"0,2","objects":1790}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11652","reported_seq":11895,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.902558+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:57.902558+0000","last_peered":"2024-09-16T04:53:57.902558+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:57.902558+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34108626,"num_objects":8329,"num_object_clones":0,"num_object_copies":16658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11624,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1649},{"shards":"2","objects":5033}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550553+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:40.550553+0000","last_peered":"2024-09-16T04:51:40.550553+0000","last_clean":"2024-09-16T04:51:40.550553+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:40.550553+0000","last_undegraded":"2024-09-16T04:51:40.550553+0000","last_fullsized":"2024-09-16T04:51:40.550553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'11646","reported_seq":18503,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:57.864814+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:57.864814+0000","last_peered":"2024-09-16T04:53:57.864814+0000","last_clean":"2024-09-16T04:53:57.864814+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:57.864814+0000","last_undegraded":"2024-09-16T04:53:57.864814+0000","last_fullsized":"2024-09-16T04:53:57.864814+0000","mapping_epoch":46,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34127872,"num_objects":8332,"num_object_clones":0,"num_object_copies":16664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8369,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'11662","reported_seq":11761,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.925253+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:57.925253+0000","last_peered":"2024-09-16T04:53:57.925253+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:57.925253+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33645161,"num_objects":8215,"num_object_clones":0,"num_object_copies":16430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11661,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1684},{"shards":"2","objects":4808}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550571+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:40.550571+0000","last_peered":"2024-09-16T04:51:40.550571+0000","last_clean":"2024-09-16T04:51:40.550571+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:40.550571+0000","last_undegraded":"2024-09-16T04:51:40.550571+0000","last_fullsized":"2024-09-16T04:51:40.550571+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12043","reported_seq":12147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.918375+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:57.918375+0000","last_peered":"2024-09-16T04:53:57.918375+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:57.918375+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34772795,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12040,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1741},{"shards":"2","objects":4978}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11637","reported_seq":17746,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.796953+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:53.796953+0000","last_peered":"2024-09-16T04:53:53.796953+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:53.796953+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35065856,"num_objects":8561,"num_object_clones":0,"num_object_copies":17122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":337,"num_objects_unfound":0,"num_objects_dirty":8561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8391,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2988,"num_bytes_recovered":12230656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8222},{"shards":"0,2","objects":337}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:45.615397+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:45.615397+0000","last_peered":"2024-09-16T04:51:45.615397+0000","last_clean":"2024-09-16T04:51:45.615397+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:45.615397+0000","last_undegraded":"2024-09-16T04:51:45.615397+0000","last_fullsized":"2024-09-16T04:51:45.615397+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:45.615423+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:45.615423+0000","last_peered":"2024-09-16T04:51:45.615423+0000","last_clean":"2024-09-16T04:51:45.615423+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:45.615423+0000","last_undegraded":"2024-09-16T04:51:45.615423+0000","last_fullsized":"2024-09-16T04:51:45.615423+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'11951","reported_seq":12088,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:53.739016+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:53.739016+0000","last_peered":"2024-09-16T04:53:53.739016+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:53.739016+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34673257,"num_objects":8466,"num_object_clones":0,"num_object_copies":16932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8720,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4942},{"shards":"0,2","objects":1787}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":549445879,"num_objects":134149,"num_object_clones":0,"num_object_copies":268298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":337,"num_objects_unfound":0,"num_objects_dirty":134149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152481,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6343,"num_bytes_recovered":25939968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1193828352,"data_stored":1193747149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160903,"ondisk_log_size":160903,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754540,"kb_used_data":416028,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93617300,"statfs":{"total":96636764160,"available":95864115200,"internally_reserved":0,"allocated":426012672,"data_stored":425739617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593781,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620864,"kb_used_data":332336,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93750976,"statfs":{"total":96636764160,"available":96000999424,"internally_reserved":0,"allocated":340312064,"data_stored":339976387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":765584,"kb_used_data":418812,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93606256,"statfs":{"total":96636764160,"available":95852806144,"internally_reserved":0,"allocated":428863488,"data_stored":428594505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,3,1,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":428437504,"data_stored":428434025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339865600,"data_stored":339795427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":425525248,"data_stored":425517697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:01.641 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:02.426 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:02.433 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:02.481 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-16T04:54:02.384916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":544968970,"num_objects":133058,"num_object_clones":0,"num_object_copies":266116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":178,"num_objects_unfound":0,"num_objects_dirty":133058,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":153576,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6503,"num_bytes_recovered":26591251,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2118456,"kb_used_data":1161280,"kb_used_omap":12,"kb_used_meta":957107,"kb_avail":280997064,"statfs":{"total":289910292480,"available":287740993536,"internally_reserved":0,"allocated":1189150720,"data_stored":1188273005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":980077569},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,2,13,4,2,9,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18567168,"num_objects":-4533,"num_object_clones":0,"num_object_copies":-9066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-306,"num_objects_unfound":0,"num_objects_dirty":-4533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":1253376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002859"},"pg_stats":[{"pgid":"2.f","version":"47'11621","reported_seq":11701,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:58.796856+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:58.796856+0000","last_peered":"2024-09-16T04:53:58.796856+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:58.796856+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33247232,"num_objects":8117,"num_object_clones":0,"num_object_copies":16234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11621,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4776},{"shards":"0,2","objects":1588}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11732","reported_seq":11849,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:57.881400+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:57.881400+0000","last_peered":"2024-09-16T04:53:57.881400+0000","last_clean":"2024-09-16T04:53:57.881400+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:57.881400+0000","last_undegraded":"2024-09-16T04:53:57.881400+0000","last_fullsized":"2024-09-16T04:53:57.881400+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33841152,"num_objects":8262,"num_object_clones":0,"num_object_copies":16524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8701,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'11606","reported_seq":11746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.891770+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:57.891770+0000","last_peered":"2024-09-16T04:53:57.891770+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:57.891770+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33796120,"num_objects":8252,"num_object_clones":0,"num_object_copies":16504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8252,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8345,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1644},{"shards":"2","objects":4933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11898","reported_seq":11969,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:57.903824+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:57.903824+0000","last_peered":"2024-09-16T04:53:57.903824+0000","last_clean":"2024-09-16T04:53:57.903824+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:57.903824+0000","last_undegraded":"2024-09-16T04:53:57.903824+0000","last_fullsized":"2024-09-16T04:53:57.903824+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34594816,"num_objects":8446,"num_object_clones":0,"num_object_copies":16892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11898,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11784","reported_seq":12003,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:58.837887+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:58.837887+0000","last_peered":"2024-09-16T04:53:58.837887+0000","last_clean":"2024-09-16T04:53:58.837887+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:58.837887+0000","last_undegraded":"2024-09-16T04:53:58.837887+0000","last_fullsized":"2024-09-16T04:53:58.837887+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34242560,"num_objects":8360,"num_object_clones":0,"num_object_copies":16720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8816,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11901","reported_seq":12035,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:58.783056+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:58.783056+0000","last_peered":"2024-09-16T04:53:58.783056+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:58.783056+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34455552,"num_objects":8412,"num_object_clones":0,"num_object_copies":16824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8894,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4956},{"shards":"0,2","objects":1714}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11710","reported_seq":11859,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:58.718125+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:58.718125+0000","last_peered":"2024-09-16T04:53:58.718125+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:58.718125+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382400,"num_objects":8150,"num_object_clones":0,"num_object_copies":16300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8365,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4751},{"shards":"0,2","objects":1622}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11718","reported_seq":11935,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:58.722363+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:58.722363+0000","last_peered":"2024-09-16T04:53:58.722363+0000","last_clean":"2024-09-16T04:53:58.722363+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:58.722363+0000","last_undegraded":"2024-09-16T04:53:58.722363+0000","last_fullsized":"2024-09-16T04:53:58.722363+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33988608,"num_objects":8298,"num_object_clones":0,"num_object_copies":16596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8788,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:35.473233+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:35.473233+0000","last_peered":"2024-09-16T04:51:35.473233+0000","last_clean":"2024-09-16T04:51:35.473233+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:35.473233+0000","last_undegraded":"2024-09-16T04:51:35.473233+0000","last_fullsized":"2024-09-16T04:51:35.473233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11735","reported_seq":11963,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.951367+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:57.951367+0000","last_peered":"2024-09-16T04:53:57.951367+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:57.951367+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34185216,"num_objects":8346,"num_object_clones":0,"num_object_copies":16692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8486,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1643},{"shards":"2","objects":5025}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11943","reported_seq":12073,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:58.802774+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:58.802774+0000","last_peered":"2024-09-16T04:53:58.802774+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:58.802774+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33894400,"num_objects":8275,"num_object_clones":0,"num_object_copies":16550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8598,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4810},{"shards":"0,2","objects":1635}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11652","reported_seq":11895,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.902558+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:57.902558+0000","last_peered":"2024-09-16T04:53:57.902558+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:57.902558+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34108626,"num_objects":8329,"num_object_clones":0,"num_object_copies":16658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11624,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1649},{"shards":"2","objects":5033}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550553+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:40.550553+0000","last_peered":"2024-09-16T04:51:40.550553+0000","last_clean":"2024-09-16T04:51:40.550553+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:40.550553+0000","last_undegraded":"2024-09-16T04:51:40.550553+0000","last_fullsized":"2024-09-16T04:51:40.550553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'11646","reported_seq":18503,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:57.864814+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:57.864814+0000","last_peered":"2024-09-16T04:53:57.864814+0000","last_clean":"2024-09-16T04:53:57.864814+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:57.864814+0000","last_undegraded":"2024-09-16T04:53:57.864814+0000","last_fullsized":"2024-09-16T04:53:57.864814+0000","mapping_epoch":46,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34127872,"num_objects":8332,"num_object_clones":0,"num_object_copies":16664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8369,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'11662","reported_seq":11761,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.925253+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:57.925253+0000","last_peered":"2024-09-16T04:53:57.925253+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:57.925253+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33645161,"num_objects":8215,"num_object_clones":0,"num_object_copies":16430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11661,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1684},{"shards":"2","objects":4808}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550571+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:40.550571+0000","last_peered":"2024-09-16T04:51:40.550571+0000","last_clean":"2024-09-16T04:51:40.550571+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:40.550571+0000","last_undegraded":"2024-09-16T04:51:40.550571+0000","last_fullsized":"2024-09-16T04:51:40.550571+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12043","reported_seq":12147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.918375+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:57.918375+0000","last_peered":"2024-09-16T04:53:57.918375+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:57.918375+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34772795,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12040,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1741},{"shards":"2","objects":4978}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11758","reported_seq":18184,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:58.807494+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:58.807494+0000","last_peered":"2024-09-16T04:53:58.807494+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:58.807494+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34570240,"num_objects":8440,"num_object_clones":0,"num_object_copies":16880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":178,"num_objects_unfound":0,"num_objects_dirty":8440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8512,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3147,"num_bytes_recovered":12881920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8260},{"shards":"0,2","objects":178}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:45.615397+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:45.615397+0000","last_peered":"2024-09-16T04:51:45.615397+0000","last_clean":"2024-09-16T04:51:45.615397+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:45.615397+0000","last_undegraded":"2024-09-16T04:51:45.615397+0000","last_fullsized":"2024-09-16T04:51:45.615397+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:45.615423+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:45.615423+0000","last_peered":"2024-09-16T04:51:45.615423+0000","last_clean":"2024-09-16T04:51:45.615423+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:45.615423+0000","last_undegraded":"2024-09-16T04:51:45.615423+0000","last_fullsized":"2024-09-16T04:51:45.615423+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12087","reported_seq":12228,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:58.697086+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:58.697086+0000","last_peered":"2024-09-16T04:53:58.697086+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:58.697086+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34116201,"num_objects":8330,"num_object_clones":0,"num_object_copies":16660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8856,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4806},{"shards":"0,2","objects":1651}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":544968951,"num_objects":133056,"num_object_clones":0,"num_object_copies":266112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":178,"num_objects_unfound":0,"num_objects_dirty":133056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":153574,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6502,"num_bytes_recovered":26591232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187774464,"data_stored":1187693261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160696,"ondisk_log_size":160696,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754540,"kb_used_data":416028,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93617300,"statfs":{"total":96636764160,"available":95864115200,"internally_reserved":0,"allocated":426012672,"data_stored":425739617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593782,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612056,"kb_used_data":331848,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93759784,"statfs":{"total":96636764160,"available":96010018816,"internally_reserved":0,"allocated":339812352,"data_stored":339476675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,7,1,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751860,"kb_used_data":413404,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93619980,"statfs":{"total":96636764160,"available":95866859520,"internally_reserved":0,"allocated":423325696,"data_stored":423056713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,0,0,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":422858752,"data_stored":422855273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339390464,"data_stored":339320291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":425525248,"data_stored":425517697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:02.482 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:03.440 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:03.441 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:03.478 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-16T04:54:02.384916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":544968970,"num_objects":133058,"num_object_clones":0,"num_object_copies":266116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":178,"num_objects_unfound":0,"num_objects_dirty":133058,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":153576,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6503,"num_bytes_recovered":26591251,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2118456,"kb_used_data":1161280,"kb_used_omap":12,"kb_used_meta":957107,"kb_avail":280997064,"statfs":{"total":289910292480,"available":287740993536,"internally_reserved":0,"allocated":1189150720,"data_stored":1188273005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":980077569},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,2,13,4,2,9,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18567168,"num_objects":-4533,"num_object_clones":0,"num_object_copies":-9066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-306,"num_objects_unfound":0,"num_objects_dirty":-4533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":1253376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002859"},"pg_stats":[{"pgid":"2.f","version":"47'11621","reported_seq":11701,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:58.796856+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:53:58.796856+0000","last_peered":"2024-09-16T04:53:58.796856+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:53:58.796856+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33247232,"num_objects":8117,"num_object_clones":0,"num_object_copies":16234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11621,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4776},{"shards":"0,2","objects":1588}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11732","reported_seq":11849,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:57.881400+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:53:57.881400+0000","last_peered":"2024-09-16T04:53:57.881400+0000","last_clean":"2024-09-16T04:53:57.881400+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:53:57.881400+0000","last_undegraded":"2024-09-16T04:53:57.881400+0000","last_fullsized":"2024-09-16T04:53:57.881400+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33841152,"num_objects":8262,"num_object_clones":0,"num_object_copies":16524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8701,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'11606","reported_seq":11746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.891770+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:53:57.891770+0000","last_peered":"2024-09-16T04:53:57.891770+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:53:57.891770+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33796120,"num_objects":8252,"num_object_clones":0,"num_object_copies":16504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8252,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8345,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1644},{"shards":"2","objects":4933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'11898","reported_seq":11969,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:57.903824+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:53:57.903824+0000","last_peered":"2024-09-16T04:53:57.903824+0000","last_clean":"2024-09-16T04:53:57.903824+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:53:57.903824+0000","last_undegraded":"2024-09-16T04:53:57.903824+0000","last_fullsized":"2024-09-16T04:53:57.903824+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34594816,"num_objects":8446,"num_object_clones":0,"num_object_copies":16892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11898,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11784","reported_seq":12003,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:58.837887+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:53:58.837887+0000","last_peered":"2024-09-16T04:53:58.837887+0000","last_clean":"2024-09-16T04:53:58.837887+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:53:58.837887+0000","last_undegraded":"2024-09-16T04:53:58.837887+0000","last_fullsized":"2024-09-16T04:53:58.837887+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34242560,"num_objects":8360,"num_object_clones":0,"num_object_copies":16720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8816,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'11901","reported_seq":12035,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:58.783056+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:53:58.783056+0000","last_peered":"2024-09-16T04:53:58.783056+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:53:58.783056+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34455552,"num_objects":8412,"num_object_clones":0,"num_object_copies":16824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8894,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4956},{"shards":"0,2","objects":1714}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11710","reported_seq":11859,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:58.718125+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:53:58.718125+0000","last_peered":"2024-09-16T04:53:58.718125+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:53:58.718125+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33382400,"num_objects":8150,"num_object_clones":0,"num_object_copies":16300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8365,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4751},{"shards":"0,2","objects":1622}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11718","reported_seq":11935,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:58.722363+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:53:58.722363+0000","last_peered":"2024-09-16T04:53:58.722363+0000","last_clean":"2024-09-16T04:53:58.722363+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:53:58.722363+0000","last_undegraded":"2024-09-16T04:53:58.722363+0000","last_fullsized":"2024-09-16T04:53:58.722363+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33988608,"num_objects":8298,"num_object_clones":0,"num_object_copies":16596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8788,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:35.473233+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:35.473233+0000","last_peered":"2024-09-16T04:51:35.473233+0000","last_clean":"2024-09-16T04:51:35.473233+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:35.473233+0000","last_undegraded":"2024-09-16T04:51:35.473233+0000","last_fullsized":"2024-09-16T04:51:35.473233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11735","reported_seq":11963,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.951367+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:53:57.951367+0000","last_peered":"2024-09-16T04:53:57.951367+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:53:57.951367+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34185216,"num_objects":8346,"num_object_clones":0,"num_object_copies":16692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8486,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1643},{"shards":"2","objects":5025}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'11943","reported_seq":12073,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:58.802774+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:53:58.802774+0000","last_peered":"2024-09-16T04:53:58.802774+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:53:58.802774+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33894400,"num_objects":8275,"num_object_clones":0,"num_object_copies":16550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8598,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4810},{"shards":"0,2","objects":1635}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11652","reported_seq":11895,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.902558+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:53:57.902558+0000","last_peered":"2024-09-16T04:53:57.902558+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:53:57.902558+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34108626,"num_objects":8329,"num_object_clones":0,"num_object_copies":16658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11624,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1649},{"shards":"2","objects":5033}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550553+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:40.550553+0000","last_peered":"2024-09-16T04:51:40.550553+0000","last_clean":"2024-09-16T04:51:40.550553+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:40.550553+0000","last_undegraded":"2024-09-16T04:51:40.550553+0000","last_fullsized":"2024-09-16T04:51:40.550553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'11646","reported_seq":18503,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:53:57.864814+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:53:57.864814+0000","last_peered":"2024-09-16T04:53:57.864814+0000","last_clean":"2024-09-16T04:53:57.864814+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:53:57.864814+0000","last_undegraded":"2024-09-16T04:53:57.864814+0000","last_fullsized":"2024-09-16T04:53:57.864814+0000","mapping_epoch":46,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34127872,"num_objects":8332,"num_object_clones":0,"num_object_copies":16664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8369,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'11662","reported_seq":11761,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.925253+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:53:57.925253+0000","last_peered":"2024-09-16T04:53:57.925253+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:53:57.925253+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33645161,"num_objects":8215,"num_object_clones":0,"num_object_copies":16430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11661,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1684},{"shards":"2","objects":4808}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550571+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:40.550571+0000","last_peered":"2024-09-16T04:51:40.550571+0000","last_clean":"2024-09-16T04:51:40.550571+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:40.550571+0000","last_undegraded":"2024-09-16T04:51:40.550571+0000","last_fullsized":"2024-09-16T04:51:40.550571+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12043","reported_seq":12147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:57.918375+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:53:57.918375+0000","last_peered":"2024-09-16T04:53:57.918375+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:53:57.918375+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34772795,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12040,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1741},{"shards":"2","objects":4978}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11758","reported_seq":18184,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:58.807494+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:53:58.807494+0000","last_peered":"2024-09-16T04:53:58.807494+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:53:58.807494+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34570240,"num_objects":8440,"num_object_clones":0,"num_object_copies":16880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":178,"num_objects_unfound":0,"num_objects_dirty":8440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8512,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3147,"num_bytes_recovered":12881920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8260},{"shards":"0,2","objects":178}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:45.615397+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:45.615397+0000","last_peered":"2024-09-16T04:51:45.615397+0000","last_clean":"2024-09-16T04:51:45.615397+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:45.615397+0000","last_undegraded":"2024-09-16T04:51:45.615397+0000","last_fullsized":"2024-09-16T04:51:45.615397+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:45.615423+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:45.615423+0000","last_peered":"2024-09-16T04:51:45.615423+0000","last_clean":"2024-09-16T04:51:45.615423+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:45.615423+0000","last_undegraded":"2024-09-16T04:51:45.615423+0000","last_fullsized":"2024-09-16T04:51:45.615423+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12087","reported_seq":12228,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:53:58.697086+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:53:58.697086+0000","last_peered":"2024-09-16T04:53:58.697086+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:53:58.697086+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34116201,"num_objects":8330,"num_object_clones":0,"num_object_copies":16660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8856,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4806},{"shards":"0,2","objects":1651}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":544968951,"num_objects":133056,"num_object_clones":0,"num_object_copies":266112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":178,"num_objects_unfound":0,"num_objects_dirty":133056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":153574,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6502,"num_bytes_recovered":26591232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187774464,"data_stored":1187693261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160696,"ondisk_log_size":160696,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754540,"kb_used_data":416028,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93617300,"statfs":{"total":96636764160,"available":95864115200,"internally_reserved":0,"allocated":426012672,"data_stored":425739617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593782,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612056,"kb_used_data":331848,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93759784,"statfs":{"total":96636764160,"available":96010018816,"internally_reserved":0,"allocated":339812352,"data_stored":339476675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,7,1,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751860,"kb_used_data":413404,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93619980,"statfs":{"total":96636764160,"available":95866859520,"internally_reserved":0,"allocated":423325696,"data_stored":423056713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,0,0,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":422858752,"data_stored":422855273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339390464,"data_stored":339320291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":425525248,"data_stored":425517697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:03.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:54:03.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'11621", 'reported_seq': 11701, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:58.796856+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:53:58.796856+0000', 'last_peered': '2024-09-16T04:53:58.796856+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:53:58.796856+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33247232, 'num_objects': 8117, 'num_object_clones': 0, 'num_object_copies': 16234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8117, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11621, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4776}, {'shards': '0,2', 'objects': 1588}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:03.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:54:03.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'11606", 'reported_seq': 11746, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:57.891770+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:53:57.891770+0000', 'last_peered': '2024-09-16T04:53:57.891770+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:53:57.891770+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33796120, 'num_objects': 8252, 'num_object_clones': 0, 'num_object_copies': 16504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8252, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8345, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1644}, {'shards': '2', 'objects': 4933}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:03.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:54:03.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'11901", 'reported_seq': 12035, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:58.783056+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:53:58.783056+0000', 'last_peered': '2024-09-16T04:53:58.783056+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:53:58.783056+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34455552, 'num_objects': 8412, 'num_object_clones': 0, 'num_object_copies': 16824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8412, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8894, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4956}, {'shards': '0,2', 'objects': 1714}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:03.480 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:54:03.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'11710", 'reported_seq': 11859, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:58.718125+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:53:58.718125+0000', 'last_peered': '2024-09-16T04:53:58.718125+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:53:58.718125+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33382400, 'num_objects': 8150, 'num_object_clones': 0, 'num_object_copies': 16300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8150, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8365, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4751}, {'shards': '0,2', 'objects': 1622}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:03.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:54:03.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'11735", 'reported_seq': 11963, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:57.951367+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:53:57.951367+0000', 'last_peered': '2024-09-16T04:53:57.951367+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:53:57.951367+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34185216, 'num_objects': 8346, 'num_object_clones': 0, 'num_object_copies': 16692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8346, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8486, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1643}, {'shards': '2', 'objects': 5025}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:03.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:54:03.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'11943", 'reported_seq': 12073, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:58.802774+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:53:58.802774+0000', 'last_peered': '2024-09-16T04:53:58.802774+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:53:58.802774+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33894400, 'num_objects': 8275, 'num_object_clones': 0, 'num_object_copies': 16550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8275, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8598, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4810}, {'shards': '0,2', 'objects': 1635}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:03.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:54:03.481 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'11652", 'reported_seq': 11895, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:57.902558+0000', 'last_change': '2024-09-16T04:49:38.112422+0000', 'last_active': '2024-09-16T04:53:57.902558+0000', 'last_peered': '2024-09-16T04:53:57.902558+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:53:57.902558+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34108626, 'num_objects': 8329, 'num_object_clones': 0, 'num_object_copies': 16658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3298, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8329, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11624, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1649}, {'shards': '2', 'objects': 5033}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:03.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:54:03.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'11662", 'reported_seq': 11761, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:57.925253+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:53:57.925253+0000', 'last_peered': '2024-09-16T04:53:57.925253+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:53:57.925253+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33645161, 'num_objects': 8215, 'num_object_clones': 0, 'num_object_copies': 16430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8215, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11661, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1684}, {'shards': '2', 'objects': 4808}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:03.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:54:03.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "47'12043", 'reported_seq': 12147, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:57.918375+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:53:57.918375+0000', 'last_peered': '2024-09-16T04:53:57.918375+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:53:57.918375+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34772795, 'num_objects': 8492, 'num_object_clones': 0, 'num_object_copies': 16984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12040, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1741}, {'shards': '2', 'objects': 4978}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:03.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-16T04:54:03.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'11758", 'reported_seq': 18184, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:58.807494+0000', 'last_change': '2024-09-16T04:51:31.766748+0000', 'last_active': '2024-09-16T04:53:58.807494+0000', 'last_peered': '2024-09-16T04:53:58.807494+0000', 'last_clean': '2024-09-16T04:49:35.635891+0000', 'last_became_active': '2024-09-16T04:49:38.175955+0000', 'last_became_peered': '2024-09-16T04:49:38.175955+0000', 'last_unstale': '2024-09-16T04:53:58.807494+0000', 'last_undegraded': '2024-09-16T04:49:37.755498+0000', 'last_fullsized': '2024-09-16T04:49:37.645966+0000', 'mapping_epoch': 44, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3255", 'last_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.024725+0000', 'objects_scrubbed': 3242, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34570240, 'num_objects': 8440, 'num_object_clones': 0, 'num_object_copies': 16880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2, 'num_objects_misplaced': 178, 'num_objects_unfound': 0, 'num_objects_dirty': 8440, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8512, 'num_write_kb': 27420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3147, 'num_bytes_recovered': 12881920, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 8260}, {'shards': '0,2', 'objects': 178}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:03.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:54:03.483 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'12087", 'reported_seq': 12228, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:53:58.697086+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:53:58.697086+0000', 'last_peered': '2024-09-16T04:53:58.697086+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:53:58.697086+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34116201, 'num_objects': 8330, 'num_object_clones': 0, 'num_object_copies': 16660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8330, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8856, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4806}, {'shards': '0,2', 'objects': 1651}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:03.483 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:54:04.459 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:04.459 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34062/266116 objects degraded (12.800%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":392,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":133058,"data_bytes":544968970,"bytes_used":2169298944,"bytes_avail":287740993536,"bytes_total":289910292480,"degraded_objects":34062,"degraded_total":266116,"degraded_ratio":0.1279968134197117,"misplaced_objects":178,"misplaced_total":266116,"misplaced_ratio":0.00066888123976010464,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":104423,"recovering_keys_per_sec":0,"num_objects_recovered":306,"num_bytes_recovered":1253376,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":377},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:54:04.566 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:54:04.566 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:05.165 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:05.166 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:05.187 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-16T04:54:04.552835+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":535507210,"num_objects":130748,"num_object_clones":0,"num_object_copies":261496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":9,"num_objects_unfound":0,"num_objects_dirty":130748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":155886,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6672,"num_bytes_recovered":27283475,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161009,"ondisk_log_size":161009,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2115376,"kb_used_data":1149884,"kb_used_omap":12,"kb_used_meta":965427,"kb_avail":281000144,"statfs":{"total":289910292480,"available":287744147456,"internally_reserved":0,"allocated":1177481216,"data_stored":1176603501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":988597249},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,10,7,5,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18731008,"num_objects":-4573,"num_object_clones":0,"num_object_copies":-9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-328,"num_objects_unfound":0,"num_objects_dirty":-4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4573,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":1343488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.169797"},"pg_stats":[{"pgid":"2.f","version":"47'11780","reported_seq":11860,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.845509+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:03.845509+0000","last_peered":"2024-09-16T04:54:03.845509+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:03.845509+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32595968,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11780,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4617},{"shards":"0,2","objects":1429}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'11866","reported_seq":11983,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:54:02.894498+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:02.894498+0000","last_peered":"2024-09-16T04:54:02.894498+0000","last_clean":"2024-09-16T04:54:02.894498+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:02.894498+0000","last_undegraded":"2024-09-16T04:54:02.894498+0000","last_fullsized":"2024-09-16T04:54:02.894498+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33292288,"num_objects":8128,"num_object_clones":0,"num_object_copies":16256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8835,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'11750","reported_seq":11890,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:02.969888+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:02.969888+0000","last_peered":"2024-09-16T04:54:02.969888+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:02.969888+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33206296,"num_objects":8108,"num_object_clones":0,"num_object_copies":16216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8108,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8489,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1500},{"shards":"2","objects":4789}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'12053","reported_seq":12124,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:54:02.945722+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:02.945722+0000","last_peered":"2024-09-16T04:54:02.945722+0000","last_clean":"2024-09-16T04:54:02.945722+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:02.945722+0000","last_undegraded":"2024-09-16T04:54:02.945722+0000","last_fullsized":"2024-09-16T04:54:02.945722+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33959936,"num_objects":8291,"num_object_clones":0,"num_object_copies":16582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12053,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11925","reported_seq":12145,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:54:03.823931+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:03.823931+0000","last_peered":"2024-09-16T04:54:03.823931+0000","last_clean":"2024-09-16T04:54:03.823931+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:03.823931+0000","last_undegraded":"2024-09-16T04:54:03.823931+0000","last_fullsized":"2024-09-16T04:54:03.823931+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33665024,"num_objects":8219,"num_object_clones":0,"num_object_copies":16438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8957,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12076","reported_seq":12210,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.824869+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:03.824869+0000","last_peered":"2024-09-16T04:54:03.824869+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:03.824869+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33738752,"num_objects":8237,"num_object_clones":0,"num_object_copies":16474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9069,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4781},{"shards":"0,2","objects":1539}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11854","reported_seq":12003,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.852252+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:03.852252+0000","last_peered":"2024-09-16T04:54:03.852252+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:03.852252+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32792576,"num_objects":8006,"num_object_clones":0,"num_object_copies":16012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8509,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4607},{"shards":"0,2","objects":1478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11847","reported_seq":12063,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:54:03.835743+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:03.835743+0000","last_peered":"2024-09-16T04:54:03.835743+0000","last_clean":"2024-09-16T04:54:03.835743+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:03.835743+0000","last_undegraded":"2024-09-16T04:54:03.835743+0000","last_fullsized":"2024-09-16T04:54:03.835743+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33460224,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8917,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:35.473233+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:51:35.473233+0000","last_peered":"2024-09-16T04:51:35.473233+0000","last_clean":"2024-09-16T04:51:35.473233+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:51:35.473233+0000","last_undegraded":"2024-09-16T04:51:35.473233+0000","last_fullsized":"2024-09-16T04:51:35.473233+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'11855","reported_seq":12084,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:02.745513+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:02.745513+0000","last_peered":"2024-09-16T04:54:02.745513+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:02.745513+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33693696,"num_objects":8226,"num_object_clones":0,"num_object_copies":16452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8606,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1523},{"shards":"2","objects":4905}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.149569+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:51:33.149569+0000","last_peered":"2024-09-16T04:51:33.149569+0000","last_clean":"2024-09-16T04:51:33.149569+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:51:33.149569+0000","last_undegraded":"2024-09-16T04:51:33.149569+0000","last_fullsized":"2024-09-16T04:51:33.149569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12107","reported_seq":12236,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.774094+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:03.774094+0000","last_peered":"2024-09-16T04:54:03.774094+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:03.774094+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33222656,"num_objects":8111,"num_object_clones":0,"num_object_copies":16222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8762,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4646},{"shards":"0,2","objects":1471}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'11771","reported_seq":12014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:02.952185+0000","last_change":"2024-09-16T04:49:38.112422+0000","last_active":"2024-09-16T04:54:02.952185+0000","last_peered":"2024-09-16T04:54:02.952185+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:02.952185+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33621202,"num_objects":8210,"num_object_clones":0,"num_object_copies":16420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11743,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1530},{"shards":"2","objects":4914}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550553+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:51:40.550553+0000","last_peered":"2024-09-16T04:51:40.550553+0000","last_clean":"2024-09-16T04:51:40.550553+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:51:40.550553+0000","last_undegraded":"2024-09-16T04:51:40.550553+0000","last_fullsized":"2024-09-16T04:51:40.550553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'11797","reported_seq":18654,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:54:02.780852+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:02.780852+0000","last_peered":"2024-09-16T04:54:02.780852+0000","last_clean":"2024-09-16T04:54:02.780852+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:02.780852+0000","last_undegraded":"2024-09-16T04:54:02.780852+0000","last_fullsized":"2024-09-16T04:54:02.780852+0000","mapping_epoch":46,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33509376,"num_objects":8181,"num_object_clones":0,"num_object_copies":16362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8520,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'11798","reported_seq":11897,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:02.955514+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:02.955514+0000","last_peered":"2024-09-16T04:54:02.955514+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:02.955514+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33088105,"num_objects":8079,"num_object_clones":0,"num_object_copies":16158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11797,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1548},{"shards":"2","objects":4672}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:40.550571+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:51:40.550571+0000","last_peered":"2024-09-16T04:51:40.550571+0000","last_clean":"2024-09-16T04:51:40.550571+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:51:40.550571+0000","last_undegraded":"2024-09-16T04:51:40.550571+0000","last_fullsized":"2024-09-16T04:51:40.550571+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12191","reported_seq":12295,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:02.950196+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:02.950196+0000","last_peered":"2024-09-16T04:54:02.950196+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:02.950196+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34166587,"num_objects":8344,"num_object_clones":0,"num_object_copies":16688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12188,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1593},{"shards":"2","objects":4830}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.153865+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:51:33.153865+0000","last_peered":"2024-09-16T04:51:33.153865+0000","last_clean":"2024-09-16T04:51:33.153865+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:51:33.153865+0000","last_undegraded":"2024-09-16T04:51:33.153865+0000","last_fullsized":"2024-09-16T04:51:33.153865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'11892","reported_seq":18655,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.848472+0000","last_change":"2024-09-16T04:51:31.766748+0000","last_active":"2024-09-16T04:54:03.848472+0000","last_peered":"2024-09-16T04:54:03.848472+0000","last_clean":"2024-09-16T04:49:35.635891+0000","last_became_active":"2024-09-16T04:49:38.175955+0000","last_became_peered":"2024-09-16T04:49:38.175955+0000","last_unstale":"2024-09-16T04:54:03.848472+0000","last_undegraded":"2024-09-16T04:49:37.755498+0000","last_fullsized":"2024-09-16T04:49:37.645966+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34021376,"num_objects":8306,"num_object_clones":0,"num_object_copies":16612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":9,"num_objects_unfound":0,"num_objects_dirty":8306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8646,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3316,"num_bytes_recovered":13574144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":8295},{"shards":"0,2","objects":9}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:45.615397+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:51:45.615397+0000","last_peered":"2024-09-16T04:51:45.615397+0000","last_clean":"2024-09-16T04:51:45.615397+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:51:45.615397+0000","last_undegraded":"2024-09-16T04:51:45.615397+0000","last_fullsized":"2024-09-16T04:51:45.615397+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:45.615423+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:51:45.615423+0000","last_peered":"2024-09-16T04:51:45.615423+0000","last_clean":"2024-09-16T04:51:45.615423+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:51:45.615423+0000","last_undegraded":"2024-09-16T04:51:45.615423+0000","last_fullsized":"2024-09-16T04:51:45.615423+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12244","reported_seq":12385,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.818461+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:03.818461+0000","last_peered":"2024-09-16T04:54:03.818461+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:03.818461+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33473129,"num_objects":8173,"num_object_clones":0,"num_object_copies":16346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9013,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4649},{"shards":"0,2","objects":1494}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":535507191,"num_objects":130746,"num_object_clones":0,"num_object_copies":261492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34062,"num_objects_misplaced":9,"num_objects_unfound":0,"num_objects_dirty":130746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":155884,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6671,"num_bytes_recovered":27283456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1175965696,"data_stored":1175884493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161006,"ondisk_log_size":161006,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":753156,"kb_used_data":410484,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93618684,"statfs":{"total":96636764160,"available":95865532416,"internally_reserved":0,"allocated":420335616,"data_stored":420062561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593782,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612056,"kb_used_data":331848,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93759784,"statfs":{"total":96636764160,"available":96010018816,"internally_reserved":0,"allocated":339812352,"data_stored":339476675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,7,1,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":750164,"kb_used_data":407552,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93621676,"statfs":{"total":96636764160,"available":95868596224,"internally_reserved":0,"allocated":417333248,"data_stored":417064265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":416747520,"data_stored":416744041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339390464,"data_stored":339320291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":419827712,"data_stored":419820161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:08.189 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:09.163 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:09.230 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-16T04:54:08.388852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":530968842,"num_objects":129640,"num_object_clones":0,"num_object_copies":259280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32571,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":129640,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156994,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6791,"num_bytes_recovered":27762707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160817,"ondisk_log_size":160817,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2118216,"kb_used_data":1144396,"kb_used_omap":12,"kb_used_meta":973747,"kb_avail":280997304,"statfs":{"total":289910292480,"available":287741239296,"internally_reserved":0,"allocated":1171861504,"data_stored":1170970205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":997116929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,1,2,13,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14000128,"num_objects":-3418,"num_object_clones":0,"num_object_copies":-6836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1491,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":-3418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3418,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1171456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004416"},"pg_stats":[{"pgid":"2.f","version":"47'11780","reported_seq":11860,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.845509+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:03.845509+0000","last_peered":"2024-09-16T04:54:03.845509+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:03.845509+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32595968,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11780,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4617},{"shards":"0,2","objects":1429}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'11989","reported_seq":12110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:07.965146+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:07.965146+0000","last_peered":"2024-09-16T04:54:07.965146+0000","last_clean":"2024-09-16T04:54:07.965146+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:07.965146+0000","last_undegraded":"2024-09-16T04:54:07.965146+0000","last_fullsized":"2024-09-16T04:54:07.965146+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32788480,"num_objects":8005,"num_object_clones":0,"num_object_copies":16010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8958,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'11890","reported_seq":12034,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.000441+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:08.000441+0000","last_peered":"2024-09-16T04:54:08.000441+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:08.000441+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32632856,"num_objects":7968,"num_object_clones":0,"num_object_copies":15936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8629,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1360},{"shards":"2","objects":4649}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12164","reported_seq":12239,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:08.039051+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:08.039051+0000","last_peered":"2024-09-16T04:54:08.039051+0000","last_clean":"2024-09-16T04:54:08.039051+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:08.039051+0000","last_undegraded":"2024-09-16T04:54:08.039051+0000","last_fullsized":"2024-09-16T04:54:08.039051+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505280,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12164,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11925","reported_seq":12145,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:54:03.823931+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:03.823931+0000","last_peered":"2024-09-16T04:54:03.823931+0000","last_clean":"2024-09-16T04:54:03.823931+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:03.823931+0000","last_undegraded":"2024-09-16T04:54:03.823931+0000","last_fullsized":"2024-09-16T04:54:03.823931+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33665024,"num_objects":8219,"num_object_clones":0,"num_object_copies":16438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8957,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12076","reported_seq":12210,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.824869+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:03.824869+0000","last_peered":"2024-09-16T04:54:03.824869+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:03.824869+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33738752,"num_objects":8237,"num_object_clones":0,"num_object_copies":16474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9069,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4781},{"shards":"0,2","objects":1539}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11854","reported_seq":12003,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.852252+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:03.852252+0000","last_peered":"2024-09-16T04:54:03.852252+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:03.852252+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32792576,"num_objects":8006,"num_object_clones":0,"num_object_copies":16012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8509,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4607},{"shards":"0,2","objects":1478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11847","reported_seq":12063,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:54:03.835743+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:03.835743+0000","last_peered":"2024-09-16T04:54:03.835743+0000","last_clean":"2024-09-16T04:54:03.835743+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:03.835743+0000","last_undegraded":"2024-09-16T04:54:03.835743+0000","last_fullsized":"2024-09-16T04:54:03.835743+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33460224,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8917,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'11990","reported_seq":12223,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.053302+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:08.053302+0000","last_peered":"2024-09-16T04:54:08.053302+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:08.053302+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33140736,"num_objects":8091,"num_object_clones":0,"num_object_copies":16182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8741,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1388},{"shards":"2","objects":4770}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12107","reported_seq":12236,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.774094+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:03.774094+0000","last_peered":"2024-09-16T04:54:03.774094+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:03.774094+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33222656,"num_objects":8111,"num_object_clones":0,"num_object_copies":16222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8762,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4646},{"shards":"0,2","objects":1471}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'11903","reported_seq":12367,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.064323+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:08.064323+0000","last_peered":"2024-09-16T04:54:08.064323+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:08.064323+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33080530,"num_objects":8078,"num_object_clones":0,"num_object_copies":16156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1809,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":8078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11875,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":434176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1381},{"shards":"2","objects":4888}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'11917","reported_seq":18778,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:07.968738+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:07.968738+0000","last_peered":"2024-09-16T04:54:07.968738+0000","last_clean":"2024-09-16T04:54:07.968738+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:07.968738+0000","last_undegraded":"2024-09-16T04:54:07.968738+0000","last_fullsized":"2024-09-16T04:54:07.968738+0000","mapping_epoch":46,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33017856,"num_objects":8061,"num_object_clones":0,"num_object_copies":16122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8640,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'11925","reported_seq":12028,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.031309+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:08.031309+0000","last_peered":"2024-09-16T04:54:08.031309+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:08.031309+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32567913,"num_objects":7952,"num_object_clones":0,"num_object_copies":15904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11924,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1421},{"shards":"2","objects":4545}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'12319","reported_seq":12427,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.000471+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:08.000471+0000","last_peered":"2024-09-16T04:54:08.000471+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:08.000471+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33642299,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12316,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1465},{"shards":"2","objects":4702}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'11984","reported_seq":18783,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:07.242504+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:07.242504+0000","last_peered":"2024-09-16T04:54:07.242504+0000","last_clean":"2024-09-16T04:54:07.242504+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:07.242504+0000","last_undegraded":"2024-09-16T04:54:07.242504+0000","last_fullsized":"2024-09-16T04:54:07.242504+0000","mapping_epoch":48,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33644544,"num_objects":8214,"num_object_clones":0,"num_object_copies":16428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8738,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12244","reported_seq":12385,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.818461+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:03.818461+0000","last_peered":"2024-09-16T04:54:03.818461+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:03.818461+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33473129,"num_objects":8173,"num_object_clones":0,"num_object_copies":16346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9013,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4649},{"shards":"0,2","objects":1494}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":530968823,"num_objects":129638,"num_object_clones":0,"num_object_copies":259276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32571,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":129638,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156992,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6790,"num_bytes_recovered":27762688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1170042880,"data_stored":1169968635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160814,"ondisk_log_size":160814,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752372,"kb_used_data":405540,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93619468,"statfs":{"total":96636764160,"available":95866335232,"internally_reserved":0,"allocated":415272960,"data_stored":414989634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,1,6,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593783,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615680,"kb_used_data":331304,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93756160,"statfs":{"total":96636764160,"available":96006307840,"internally_reserved":0,"allocated":339255296,"data_stored":338916306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,1,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":750164,"kb_used_data":407552,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93621676,"statfs":{"total":96636764160,"available":95868596224,"internally_reserved":0,"allocated":417333248,"data_stored":417064265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":416747520,"data_stored":416744041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338653184,"data_stored":338589969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414642176,"data_stored":414634625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:09.232 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:10.119 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:10.119 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:10.162 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-16T04:54:08.388852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":530968842,"num_objects":129640,"num_object_clones":0,"num_object_copies":259280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32571,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":129640,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156994,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6791,"num_bytes_recovered":27762707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160817,"ondisk_log_size":160817,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2118216,"kb_used_data":1144396,"kb_used_omap":12,"kb_used_meta":973747,"kb_avail":280997304,"statfs":{"total":289910292480,"available":287741239296,"internally_reserved":0,"allocated":1171861504,"data_stored":1170970205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":997116929},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,1,2,13,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14000128,"num_objects":-3418,"num_object_clones":0,"num_object_copies":-6836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1491,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":-3418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3418,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1171456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004416"},"pg_stats":[{"pgid":"2.f","version":"47'11780","reported_seq":11860,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.845509+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:03.845509+0000","last_peered":"2024-09-16T04:54:03.845509+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:03.845509+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32595968,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11780,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4617},{"shards":"0,2","objects":1429}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'11989","reported_seq":12110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:07.965146+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:07.965146+0000","last_peered":"2024-09-16T04:54:07.965146+0000","last_clean":"2024-09-16T04:54:07.965146+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:07.965146+0000","last_undegraded":"2024-09-16T04:54:07.965146+0000","last_fullsized":"2024-09-16T04:54:07.965146+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32788480,"num_objects":8005,"num_object_clones":0,"num_object_copies":16010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8958,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'11890","reported_seq":12034,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.000441+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:08.000441+0000","last_peered":"2024-09-16T04:54:08.000441+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:08.000441+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32632856,"num_objects":7968,"num_object_clones":0,"num_object_copies":15936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8629,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1360},{"shards":"2","objects":4649}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12164","reported_seq":12239,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:08.039051+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:08.039051+0000","last_peered":"2024-09-16T04:54:08.039051+0000","last_clean":"2024-09-16T04:54:08.039051+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:08.039051+0000","last_undegraded":"2024-09-16T04:54:08.039051+0000","last_fullsized":"2024-09-16T04:54:08.039051+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505280,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12164,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'11925","reported_seq":12145,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:54:03.823931+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:03.823931+0000","last_peered":"2024-09-16T04:54:03.823931+0000","last_clean":"2024-09-16T04:54:03.823931+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:03.823931+0000","last_undegraded":"2024-09-16T04:54:03.823931+0000","last_fullsized":"2024-09-16T04:54:03.823931+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33665024,"num_objects":8219,"num_object_clones":0,"num_object_copies":16438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8957,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12076","reported_seq":12210,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.824869+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:03.824869+0000","last_peered":"2024-09-16T04:54:03.824869+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:03.824869+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33738752,"num_objects":8237,"num_object_clones":0,"num_object_copies":16474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9069,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4781},{"shards":"0,2","objects":1539}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'11854","reported_seq":12003,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.852252+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:03.852252+0000","last_peered":"2024-09-16T04:54:03.852252+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:03.852252+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32792576,"num_objects":8006,"num_object_clones":0,"num_object_copies":16012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8509,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4607},{"shards":"0,2","objects":1478}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'11847","reported_seq":12063,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:54:03.835743+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:03.835743+0000","last_peered":"2024-09-16T04:54:03.835743+0000","last_clean":"2024-09-16T04:54:03.835743+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:03.835743+0000","last_undegraded":"2024-09-16T04:54:03.835743+0000","last_fullsized":"2024-09-16T04:54:03.835743+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33460224,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8917,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'11990","reported_seq":12223,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.053302+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:08.053302+0000","last_peered":"2024-09-16T04:54:08.053302+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:08.053302+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33140736,"num_objects":8091,"num_object_clones":0,"num_object_copies":16182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8741,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1388},{"shards":"2","objects":4770}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12107","reported_seq":12236,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.774094+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:03.774094+0000","last_peered":"2024-09-16T04:54:03.774094+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:03.774094+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33222656,"num_objects":8111,"num_object_clones":0,"num_object_copies":16222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8762,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4646},{"shards":"0,2","objects":1471}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T04:51:33.154296+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:51:33.154296+0000","last_peered":"2024-09-16T04:51:33.154296+0000","last_clean":"2024-09-16T04:51:33.154296+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:51:33.154296+0000","last_undegraded":"2024-09-16T04:51:33.154296+0000","last_fullsized":"2024-09-16T04:51:33.154296+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'11903","reported_seq":12367,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.064323+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:08.064323+0000","last_peered":"2024-09-16T04:54:08.064323+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:08.064323+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33080530,"num_objects":8078,"num_object_clones":0,"num_object_copies":16156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1809,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":8078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11875,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":434176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1381},{"shards":"2","objects":4888}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'11917","reported_seq":18778,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:07.968738+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:07.968738+0000","last_peered":"2024-09-16T04:54:07.968738+0000","last_clean":"2024-09-16T04:54:07.968738+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:07.968738+0000","last_undegraded":"2024-09-16T04:54:07.968738+0000","last_fullsized":"2024-09-16T04:54:07.968738+0000","mapping_epoch":46,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33017856,"num_objects":8061,"num_object_clones":0,"num_object_copies":16122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8640,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'11925","reported_seq":12028,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.031309+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:08.031309+0000","last_peered":"2024-09-16T04:54:08.031309+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:08.031309+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32567913,"num_objects":7952,"num_object_clones":0,"num_object_copies":15904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11924,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1421},{"shards":"2","objects":4545}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'12319","reported_seq":12427,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.000471+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:08.000471+0000","last_peered":"2024-09-16T04:54:08.000471+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:08.000471+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33642299,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12316,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1465},{"shards":"2","objects":4702}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'11984","reported_seq":18783,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:07.242504+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:07.242504+0000","last_peered":"2024-09-16T04:54:07.242504+0000","last_clean":"2024-09-16T04:54:07.242504+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:07.242504+0000","last_undegraded":"2024-09-16T04:54:07.242504+0000","last_fullsized":"2024-09-16T04:54:07.242504+0000","mapping_epoch":48,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33644544,"num_objects":8214,"num_object_clones":0,"num_object_copies":16428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8738,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12244","reported_seq":12385,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:03.818461+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:03.818461+0000","last_peered":"2024-09-16T04:54:03.818461+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:03.818461+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33473129,"num_objects":8173,"num_object_clones":0,"num_object_copies":16346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9013,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4649},{"shards":"0,2","objects":1494}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":530968823,"num_objects":129638,"num_object_clones":0,"num_object_copies":259276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32571,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":129638,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156992,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6790,"num_bytes_recovered":27762688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1170042880,"data_stored":1169968635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160814,"ondisk_log_size":160814,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752372,"kb_used_data":405540,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93619468,"statfs":{"total":96636764160,"available":95866335232,"internally_reserved":0,"allocated":415272960,"data_stored":414989634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,1,6,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593783,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615680,"kb_used_data":331304,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93756160,"statfs":{"total":96636764160,"available":96006307840,"internally_reserved":0,"allocated":339255296,"data_stored":338916306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,1,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":750164,"kb_used_data":407552,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93621676,"statfs":{"total":96636764160,"available":95868596224,"internally_reserved":0,"allocated":417333248,"data_stored":417064265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":416747520,"data_stored":416744041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338653184,"data_stored":338589969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414642176,"data_stored":414634625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:10.163 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:54:10.163 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'11780", 'reported_seq': 11860, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:03.845509+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:54:03.845509+0000', 'last_peered': '2024-09-16T04:54:03.845509+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:54:03.845509+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32595968, 'num_objects': 7958, 'num_object_clones': 0, 'num_object_copies': 15916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7958, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11780, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4617}, {'shards': '0,2', 'objects': 1429}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:10.163 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:54:10.164 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'11890", 'reported_seq': 12034, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:08.000441+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:54:08.000441+0000', 'last_peered': '2024-09-16T04:54:08.000441+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:54:08.000441+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32632856, 'num_objects': 7968, 'num_object_clones': 0, 'num_object_copies': 15936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7968, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8629, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1360}, {'shards': '2', 'objects': 4649}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:10.164 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:54:10.164 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'12076", 'reported_seq': 12210, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:03.824869+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:54:03.824869+0000', 'last_peered': '2024-09-16T04:54:03.824869+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:54:03.824869+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33738752, 'num_objects': 8237, 'num_object_clones': 0, 'num_object_copies': 16474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8237, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9069, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4781}, {'shards': '0,2', 'objects': 1539}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:10.164 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:54:10.164 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'11854", 'reported_seq': 12003, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:03.852252+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:54:03.852252+0000', 'last_peered': '2024-09-16T04:54:03.852252+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:54:03.852252+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32792576, 'num_objects': 8006, 'num_object_clones': 0, 'num_object_copies': 16012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8006, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8509, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4607}, {'shards': '0,2', 'objects': 1478}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:10.164 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:54:10.164 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'11990", 'reported_seq': 12223, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:08.053302+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:54:08.053302+0000', 'last_peered': '2024-09-16T04:54:08.053302+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:54:08.053302+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33140736, 'num_objects': 8091, 'num_object_clones': 0, 'num_object_copies': 16182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8091, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8741, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1388}, {'shards': '2', 'objects': 4770}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:10.164 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:54:10.164 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'12107", 'reported_seq': 12236, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:03.774094+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:54:03.774094+0000', 'last_peered': '2024-09-16T04:54:03.774094+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:54:03.774094+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33222656, 'num_objects': 8111, 'num_object_clones': 0, 'num_object_copies': 16222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8111, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8762, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4646}, {'shards': '0,2', 'objects': 1471}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:10.164 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:54:10.165 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'11903", 'reported_seq': 12367, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:08.064323+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:54:08.064323+0000', 'last_peered': '2024-09-16T04:54:08.064323+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:54:08.064323+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33080530, 'num_objects': 8078, 'num_object_clones': 0, 'num_object_copies': 16156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1809, 'num_objects_misplaced': 1381, 'num_objects_unfound': 0, 'num_objects_dirty': 8078, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11875, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 110, 'num_bytes_recovered': 434176, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1381}, {'shards': '2', 'objects': 4888}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:10.165 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:54:10.165 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'11925", 'reported_seq': 12028, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:08.031309+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:54:08.031309+0000', 'last_peered': '2024-09-16T04:54:08.031309+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:54:08.031309+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32567913, 'num_objects': 7952, 'num_object_clones': 0, 'num_object_copies': 15904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11924, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1421}, {'shards': '2', 'objects': 4545}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:10.165 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:54:10.165 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'12319", 'reported_seq': 12427, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:08.000471+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:54:08.000471+0000', 'last_peered': '2024-09-16T04:54:08.000471+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:54:08.000471+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33642299, 'num_objects': 8216, 'num_object_clones': 0, 'num_object_copies': 16432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8216, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12316, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1465}, {'shards': '2', 'objects': 4702}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:10.165 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:54:10.165 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'12244", 'reported_seq': 12385, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:03.818461+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:54:03.818461+0000', 'last_peered': '2024-09-16T04:54:03.818461+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:54:03.818461+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33473129, 'num_objects': 8173, 'num_object_clones': 0, 'num_object_copies': 16346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9013, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4649}, {'shards': '0,2', 'objects': 1494}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:10.166 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:54:11.325 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:11.325 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32571/257506 objects degraded (12.649%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":399,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":128753,"data_bytes":527335690,"bytes_used":2168156160,"bytes_avail":287742136320,"bytes_total":289910292480,"degraded_objects":32571,"degraded_total":257506,"degraded_ratio":0.12648637313305322,"misplaced_objects":1381,"misplaced_total":257506,"misplaced_ratio":0.0053629818334330085,"recovering_objects_per_sec":35,"recovering_bytes_per_sec":146350,"recovering_keys_per_sec":0,"num_objects_recovered":288,"num_bytes_recovered":1171456,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":537},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:54:11.402 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:54:11.402 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:12.132 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:12.133 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:12.225 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-16T04:54:10.389383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527335690,"num_objects":128753,"num_object_clones":0,"num_object_copies":257506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32571,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":128753,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":157881,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6791,"num_bytes_recovered":27762707,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2117340,"kb_used_data":1139364,"kb_used_omap":12,"kb_used_meta":977907,"kb_avail":280998180,"statfs":{"total":289910292480,"available":287742136320,"internally_reserved":0,"allocated":1166708736,"data_stored":1165807166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1001376769},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,1,4,11,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17633280,"num_objects":-4305,"num_object_clones":0,"num_object_copies":-8610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1491,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":-4305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4305,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":1171456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004467"},"pg_stats":[{"pgid":"2.f","version":"49'11910","reported_seq":11994,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.853797+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:08.853797+0000","last_peered":"2024-09-16T04:54:08.853797+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:08.853797+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32063488,"num_objects":7828,"num_object_clones":0,"num_object_copies":15656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11910,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4487},{"shards":"0,2","objects":1299}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'11989","reported_seq":12110,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:07.965146+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:07.965146+0000","last_peered":"2024-09-16T04:54:07.965146+0000","last_clean":"2024-09-16T04:54:07.965146+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:07.965146+0000","last_undegraded":"2024-09-16T04:54:07.965146+0000","last_fullsized":"2024-09-16T04:54:07.965146+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32788480,"num_objects":8005,"num_object_clones":0,"num_object_copies":16010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8958,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'11890","reported_seq":12034,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.000441+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:08.000441+0000","last_peered":"2024-09-16T04:54:08.000441+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:08.000441+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32632856,"num_objects":7968,"num_object_clones":0,"num_object_copies":15936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8629,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1360},{"shards":"2","objects":4649}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12164","reported_seq":12239,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:08.039051+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:08.039051+0000","last_peered":"2024-09-16T04:54:08.039051+0000","last_clean":"2024-09-16T04:54:08.039051+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:08.039051+0000","last_undegraded":"2024-09-16T04:54:08.039051+0000","last_fullsized":"2024-09-16T04:54:08.039051+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505280,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12164,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12049","reported_seq":12273,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:08.709469+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:08.709469+0000","last_peered":"2024-09-16T04:54:08.709469+0000","last_clean":"2024-09-16T04:54:08.709469+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:08.709469+0000","last_undegraded":"2024-09-16T04:54:08.709469+0000","last_fullsized":"2024-09-16T04:54:08.709469+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33157120,"num_objects":8095,"num_object_clones":0,"num_object_copies":16190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9081,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'12219","reported_seq":12357,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.749667+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:08.749667+0000","last_peered":"2024-09-16T04:54:08.749667+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:08.749667+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33153024,"num_objects":8094,"num_object_clones":0,"num_object_copies":16188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9212,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4638},{"shards":"0,2","objects":1396}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'11976","reported_seq":12129,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.795366+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:08.795366+0000","last_peered":"2024-09-16T04:54:08.795366+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:08.795366+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32292864,"num_objects":7884,"num_object_clones":0,"num_object_copies":15768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8631,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4485},{"shards":"0,2","objects":1356}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'11967","reported_seq":12188,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:08.873473+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:08.873473+0000","last_peered":"2024-09-16T04:54:08.873473+0000","last_clean":"2024-09-16T04:54:08.873473+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:08.873473+0000","last_undegraded":"2024-09-16T04:54:08.873473+0000","last_fullsized":"2024-09-16T04:54:08.873473+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32968704,"num_objects":8049,"num_object_clones":0,"num_object_copies":16098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9037,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'11990","reported_seq":12223,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.053302+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:08.053302+0000","last_peered":"2024-09-16T04:54:08.053302+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:08.053302+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33140736,"num_objects":8091,"num_object_clones":0,"num_object_copies":16182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8741,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1388},{"shards":"2","objects":4770}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'12215","reported_seq":12349,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.886221+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:08.886221+0000","last_peered":"2024-09-16T04:54:08.886221+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:08.886221+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32780288,"num_objects":8003,"num_object_clones":0,"num_object_copies":16006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8870,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4538},{"shards":"0,2","objects":1363}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556564+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556564+0000","last_peered":"2024-09-16T04:54:05.556564+0000","last_clean":"2024-09-16T04:54:05.556564+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556564+0000","last_undegraded":"2024-09-16T04:54:05.556564+0000","last_fullsized":"2024-09-16T04:54:05.556564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'11903","reported_seq":12367,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.064323+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:08.064323+0000","last_peered":"2024-09-16T04:54:08.064323+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:08.064323+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33080530,"num_objects":8078,"num_object_clones":0,"num_object_copies":16156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1809,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":8078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11875,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":434176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1381},{"shards":"2","objects":4888}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'11917","reported_seq":18778,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:07.968738+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:07.968738+0000","last_peered":"2024-09-16T04:54:07.968738+0000","last_clean":"2024-09-16T04:54:07.968738+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:07.968738+0000","last_undegraded":"2024-09-16T04:54:07.968738+0000","last_fullsized":"2024-09-16T04:54:07.968738+0000","mapping_epoch":46,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33017856,"num_objects":8061,"num_object_clones":0,"num_object_copies":16122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8640,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'11925","reported_seq":12028,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.031309+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:08.031309+0000","last_peered":"2024-09-16T04:54:08.031309+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:08.031309+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32567913,"num_objects":7952,"num_object_clones":0,"num_object_copies":15904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11924,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1421},{"shards":"2","objects":4545}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'12319","reported_seq":12427,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.000471+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:08.000471+0000","last_peered":"2024-09-16T04:54:08.000471+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:08.000471+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33642299,"num_objects":8216,"num_object_clones":0,"num_object_copies":16432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12316,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1465},{"shards":"2","objects":4702}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'11984","reported_seq":18783,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:07.242504+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:07.242504+0000","last_peered":"2024-09-16T04:54:07.242504+0000","last_clean":"2024-09-16T04:54:07.242504+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:07.242504+0000","last_undegraded":"2024-09-16T04:54:07.242504+0000","last_fullsized":"2024-09-16T04:54:07.242504+0000","mapping_epoch":48,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33644544,"num_objects":8214,"num_object_clones":0,"num_object_copies":16428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8738,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'12384","reported_seq":12529,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:08.887315+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:08.887315+0000","last_peered":"2024-09-16T04:54:08.887315+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:08.887315+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899689,"num_objects":8033,"num_object_clones":0,"num_object_copies":16066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9153,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4509},{"shards":"0,2","objects":1354}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":527335671,"num_objects":128751,"num_object_clones":0,"num_object_copies":257502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32571,"num_objects_misplaced":1381,"num_objects_unfound":0,"num_objects_dirty":128751,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":157879,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6790,"num_bytes_recovered":27762688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1164877824,"data_stored":1164803579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160801,"ondisk_log_size":160801,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":752372,"kb_used_data":405540,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93619468,"statfs":{"total":96636764160,"available":95866335232,"internally_reserved":0,"allocated":415272960,"data_stored":414989634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,1,6,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593783,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615680,"kb_used_data":331304,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93756160,"statfs":{"total":96636764160,"available":96006307840,"internally_reserved":0,"allocated":339255296,"data_stored":338916306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,1,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182474,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":749288,"kb_used_data":402520,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93622552,"statfs":{"total":96636764160,"available":95869493248,"internally_reserved":0,"allocated":412180480,"data_stored":411901226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411582464,"data_stored":411578985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":338653184,"data_stored":338589969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414642176,"data_stored":414634625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:15.228 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:16.006 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:16.081 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-16T04:54:14.390319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":518041866,"num_objects":126484,"num_object_clones":0,"num_object_copies":252968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32632,"num_objects_misplaced":1238,"num_objects_unfound":0,"num_objects_dirty":126484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160150,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6873,"num_bytes_recovered":28098579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160673,"ondisk_log_size":160673,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2088032,"kb_used_data":1126692,"kb_used_omap":12,"kb_used_meta":961267,"kb_avail":281027488,"statfs":{"total":289910292480,"available":287772147712,"internally_reserved":0,"allocated":1153732608,"data_stored":1152831038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":984337409},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,4,6,7,12,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17465344,"num_objects":-4264,"num_object_clones":0,"num_object_copies":-8528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1430,"num_objects_misplaced":1229,"num_objects_unfound":0,"num_objects_dirty":-4264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4264,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":815104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.837484"},"pg_stats":[{"pgid":"2.f","version":"49'12057","reported_seq":12139,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.886164+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:13.886164+0000","last_peered":"2024-09-16T04:54:13.886164+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:13.886164+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31461376,"num_objects":7681,"num_object_clones":0,"num_object_copies":15362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12057,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4340},{"shards":"0,2","objects":1152}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12132","reported_seq":12253,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:13.078858+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:13.078858+0000","last_peered":"2024-09-16T04:54:13.078858+0000","last_clean":"2024-09-16T04:54:13.078858+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:13.078858+0000","last_undegraded":"2024-09-16T04:54:13.078858+0000","last_fullsized":"2024-09-16T04:54:13.078858+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32202752,"num_objects":7862,"num_object_clones":0,"num_object_copies":15724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9101,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12052","reported_seq":12196,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.123780+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:13.123780+0000","last_peered":"2024-09-16T04:54:13.123780+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:13.123780+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31969304,"num_objects":7806,"num_object_clones":0,"num_object_copies":15612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7806,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8791,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1198},{"shards":"2","objects":4487}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12317","reported_seq":12392,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:13.036226+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:13.036226+0000","last_peered":"2024-09-16T04:54:13.036226+0000","last_clean":"2024-09-16T04:54:13.036226+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:13.036226+0000","last_undegraded":"2024-09-16T04:54:13.036226+0000","last_fullsized":"2024-09-16T04:54:13.036226+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32878592,"num_objects":8027,"num_object_clones":0,"num_object_copies":16054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12317,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12184","reported_seq":12408,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:13.770395+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:13.770395+0000","last_peered":"2024-09-16T04:54:13.770395+0000","last_clean":"2024-09-16T04:54:13.770395+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:13.770395+0000","last_undegraded":"2024-09-16T04:54:13.770395+0000","last_fullsized":"2024-09-16T04:54:13.770395+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32604160,"num_objects":7960,"num_object_clones":0,"num_object_copies":15920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'12341","reported_seq":12479,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.825884+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:13.825884+0000","last_peered":"2024-09-16T04:54:13.825884+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:13.825884+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32653312,"num_objects":7972,"num_object_clones":0,"num_object_copies":15944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9334,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4516},{"shards":"0,2","objects":1274}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12112","reported_seq":12265,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.841501+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:13.841501+0000","last_peered":"2024-09-16T04:54:13.841501+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:13.841501+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31735808,"num_objects":7748,"num_object_clones":0,"num_object_copies":15496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8767,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4349},{"shards":"0,2","objects":1220}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12102","reported_seq":12322,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:13.866537+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:13.866537+0000","last_peered":"2024-09-16T04:54:13.866537+0000","last_clean":"2024-09-16T04:54:13.866537+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:13.866537+0000","last_undegraded":"2024-09-16T04:54:13.866537+0000","last_fullsized":"2024-09-16T04:54:13.866537+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32415744,"num_objects":7914,"num_object_clones":0,"num_object_copies":15828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9172,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12134","reported_seq":12367,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.067724+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:13.067724+0000","last_peered":"2024-09-16T04:54:13.067724+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:13.067724+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32550912,"num_objects":7947,"num_object_clones":0,"num_object_copies":15894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8885,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1244},{"shards":"2","objects":4626}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'12343","reported_seq":12478,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.881235+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:13.881235+0000","last_peered":"2024-09-16T04:54:13.881235+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:13.881235+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32256000,"num_objects":7875,"num_object_clones":0,"num_object_copies":15750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8998,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4410},{"shards":"0,2","objects":1235}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556564+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556564+0000","last_peered":"2024-09-16T04:54:05.556564+0000","last_clean":"2024-09-16T04:54:05.556564+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556564+0000","last_undegraded":"2024-09-16T04:54:05.556564+0000","last_fullsized":"2024-09-16T04:54:05.556564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12062","reported_seq":12688,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.109431+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:13.109431+0000","last_peered":"2024-09-16T04:54:13.109431+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:13.109431+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32429266,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1870,"num_objects_misplaced":1238,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12034,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":770048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1238},{"shards":"2","objects":4811}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12053","reported_seq":18914,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:13.092608+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:13.092608+0000","last_peered":"2024-09-16T04:54:13.092608+0000","last_clean":"2024-09-16T04:54:13.092608+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:13.092608+0000","last_undegraded":"2024-09-16T04:54:13.092608+0000","last_fullsized":"2024-09-16T04:54:13.092608+0000","mapping_epoch":46,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32460800,"num_objects":7925,"num_object_clones":0,"num_object_copies":15850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8776,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12060","reported_seq":12163,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:12.990287+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:12.990287+0000","last_peered":"2024-09-16T04:54:12.990287+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:12.990287+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32014953,"num_objects":7817,"num_object_clones":0,"num_object_copies":15634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12059,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1286},{"shards":"2","objects":4410}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'12464","reported_seq":12572,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.121995+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:13.121995+0000","last_peered":"2024-09-16T04:54:13.121995+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:13.121995+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33048379,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12461,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1320},{"shards":"2","objects":4557}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12130","reported_seq":18928,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:12.292324+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:12.292324+0000","last_peered":"2024-09-16T04:54:12.292324+0000","last_clean":"2024-09-16T04:54:12.292324+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:12.292324+0000","last_undegraded":"2024-09-16T04:54:12.292324+0000","last_fullsized":"2024-09-16T04:54:12.292324+0000","mapping_epoch":48,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33046528,"num_objects":8068,"num_object_clones":0,"num_object_copies":16136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8884,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'12527","reported_seq":12672,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.863904+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:13.863904+0000","last_peered":"2024-09-16T04:54:13.863904+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:13.863904+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313961,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9296,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4366},{"shards":"0,2","objects":1211}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":518041847,"num_objects":126482,"num_object_clones":0,"num_object_copies":252964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32632,"num_objects_misplaced":1238,"num_objects_unfound":0,"num_objects_dirty":126482,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160148,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6872,"num_bytes_recovered":28098560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1152053248,"data_stored":1151979003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":734036,"kb_used_data":399676,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93637804,"statfs":{"total":96636764160,"available":95885111296,"internally_reserved":0,"allocated":409268224,"data_stored":408984898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593784,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622716,"kb_used_data":330028,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93749124,"statfs":{"total":96636764160,"available":95999102976,"internally_reserved":0,"allocated":337948672,"data_stored":337609682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731280,"kb_used_data":396988,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93640560,"statfs":{"total":96636764160,"available":95887933440,"internally_reserved":0,"allocated":406515712,"data_stored":406236458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,0,2,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405925888,"data_stored":405922409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":337387520,"data_stored":337324305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408739840,"data_stored":408732289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:16.082 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:16.870 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:16.872 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:16.958 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-16T04:54:16.392775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":518041866,"num_objects":126484,"num_object_clones":0,"num_object_copies":252968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32632,"num_objects_misplaced":1238,"num_objects_unfound":0,"num_objects_dirty":126484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160150,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6873,"num_bytes_recovered":28098579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160673,"ondisk_log_size":160673,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2088032,"kb_used_data":1126692,"kb_used_omap":12,"kb_used_meta":961267,"kb_avail":281027488,"statfs":{"total":289910292480,"available":287772147712,"internally_reserved":0,"allocated":1153732608,"data_stored":1152831038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":984337409},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,4,6,7,12,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17465344,"num_objects":-4264,"num_object_clones":0,"num_object_copies":-8528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1430,"num_objects_misplaced":1229,"num_objects_unfound":0,"num_objects_dirty":-4264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4264,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":815104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.839093"},"pg_stats":[{"pgid":"2.f","version":"49'12057","reported_seq":12139,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.886164+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:13.886164+0000","last_peered":"2024-09-16T04:54:13.886164+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:13.886164+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31461376,"num_objects":7681,"num_object_clones":0,"num_object_copies":15362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12057,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4340},{"shards":"0,2","objects":1152}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12132","reported_seq":12253,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:13.078858+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:13.078858+0000","last_peered":"2024-09-16T04:54:13.078858+0000","last_clean":"2024-09-16T04:54:13.078858+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:13.078858+0000","last_undegraded":"2024-09-16T04:54:13.078858+0000","last_fullsized":"2024-09-16T04:54:13.078858+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32202752,"num_objects":7862,"num_object_clones":0,"num_object_copies":15724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9101,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12052","reported_seq":12196,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.123780+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:13.123780+0000","last_peered":"2024-09-16T04:54:13.123780+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:13.123780+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31969304,"num_objects":7806,"num_object_clones":0,"num_object_copies":15612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7806,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8791,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1198},{"shards":"2","objects":4487}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12317","reported_seq":12392,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:13.036226+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:13.036226+0000","last_peered":"2024-09-16T04:54:13.036226+0000","last_clean":"2024-09-16T04:54:13.036226+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:13.036226+0000","last_undegraded":"2024-09-16T04:54:13.036226+0000","last_fullsized":"2024-09-16T04:54:13.036226+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32878592,"num_objects":8027,"num_object_clones":0,"num_object_copies":16054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12317,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12184","reported_seq":12408,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:13.770395+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:13.770395+0000","last_peered":"2024-09-16T04:54:13.770395+0000","last_clean":"2024-09-16T04:54:13.770395+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:13.770395+0000","last_undegraded":"2024-09-16T04:54:13.770395+0000","last_fullsized":"2024-09-16T04:54:13.770395+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32604160,"num_objects":7960,"num_object_clones":0,"num_object_copies":15920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'12341","reported_seq":12479,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.825884+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:13.825884+0000","last_peered":"2024-09-16T04:54:13.825884+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:13.825884+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32653312,"num_objects":7972,"num_object_clones":0,"num_object_copies":15944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9334,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4516},{"shards":"0,2","objects":1274}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12112","reported_seq":12265,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.841501+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:13.841501+0000","last_peered":"2024-09-16T04:54:13.841501+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:13.841501+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31735808,"num_objects":7748,"num_object_clones":0,"num_object_copies":15496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8767,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4349},{"shards":"0,2","objects":1220}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12102","reported_seq":12322,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:13.866537+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:13.866537+0000","last_peered":"2024-09-16T04:54:13.866537+0000","last_clean":"2024-09-16T04:54:13.866537+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:13.866537+0000","last_undegraded":"2024-09-16T04:54:13.866537+0000","last_fullsized":"2024-09-16T04:54:13.866537+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32415744,"num_objects":7914,"num_object_clones":0,"num_object_copies":15828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9172,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12134","reported_seq":12367,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.067724+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:13.067724+0000","last_peered":"2024-09-16T04:54:13.067724+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:13.067724+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32550912,"num_objects":7947,"num_object_clones":0,"num_object_copies":15894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8885,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1244},{"shards":"2","objects":4626}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'12343","reported_seq":12478,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.881235+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:13.881235+0000","last_peered":"2024-09-16T04:54:13.881235+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:13.881235+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32256000,"num_objects":7875,"num_object_clones":0,"num_object_copies":15750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8998,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4410},{"shards":"0,2","objects":1235}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556564+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556564+0000","last_peered":"2024-09-16T04:54:05.556564+0000","last_clean":"2024-09-16T04:54:05.556564+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556564+0000","last_undegraded":"2024-09-16T04:54:05.556564+0000","last_fullsized":"2024-09-16T04:54:05.556564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12062","reported_seq":12688,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.109431+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:13.109431+0000","last_peered":"2024-09-16T04:54:13.109431+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:13.109431+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32429266,"num_objects":7919,"num_object_clones":0,"num_object_copies":15838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1870,"num_objects_misplaced":1238,"num_objects_unfound":0,"num_objects_dirty":7919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12034,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":770048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1238},{"shards":"2","objects":4811}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12053","reported_seq":18914,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:13.092608+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:13.092608+0000","last_peered":"2024-09-16T04:54:13.092608+0000","last_clean":"2024-09-16T04:54:13.092608+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:13.092608+0000","last_undegraded":"2024-09-16T04:54:13.092608+0000","last_fullsized":"2024-09-16T04:54:13.092608+0000","mapping_epoch":46,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32460800,"num_objects":7925,"num_object_clones":0,"num_object_copies":15850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8776,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12060","reported_seq":12163,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:12.990287+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:12.990287+0000","last_peered":"2024-09-16T04:54:12.990287+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:12.990287+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32014953,"num_objects":7817,"num_object_clones":0,"num_object_copies":15634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12059,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1286},{"shards":"2","objects":4410}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'12464","reported_seq":12572,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.121995+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:13.121995+0000","last_peered":"2024-09-16T04:54:13.121995+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:13.121995+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33048379,"num_objects":8071,"num_object_clones":0,"num_object_copies":16142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12461,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1320},{"shards":"2","objects":4557}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12130","reported_seq":18928,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:12.292324+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:12.292324+0000","last_peered":"2024-09-16T04:54:12.292324+0000","last_clean":"2024-09-16T04:54:12.292324+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:12.292324+0000","last_undegraded":"2024-09-16T04:54:12.292324+0000","last_fullsized":"2024-09-16T04:54:12.292324+0000","mapping_epoch":48,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33046528,"num_objects":8068,"num_object_clones":0,"num_object_copies":16136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8884,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'12527","reported_seq":12672,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.863904+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:13.863904+0000","last_peered":"2024-09-16T04:54:13.863904+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:13.863904+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313961,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9296,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4366},{"shards":"0,2","objects":1211}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":518041847,"num_objects":126482,"num_object_clones":0,"num_object_copies":252964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32632,"num_objects_misplaced":1238,"num_objects_unfound":0,"num_objects_dirty":126482,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":160148,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6872,"num_bytes_recovered":28098560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1152053248,"data_stored":1151979003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":734036,"kb_used_data":399676,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93637804,"statfs":{"total":96636764160,"available":95885111296,"internally_reserved":0,"allocated":409268224,"data_stored":408984898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593784,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622716,"kb_used_data":330028,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93749124,"statfs":{"total":96636764160,"available":95999102976,"internally_reserved":0,"allocated":337948672,"data_stored":337609682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731280,"kb_used_data":396988,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93640560,"statfs":{"total":96636764160,"available":95887933440,"internally_reserved":0,"allocated":406515712,"data_stored":406236458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,0,2,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405925888,"data_stored":405922409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":337387520,"data_stored":337324305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408739840,"data_stored":408732289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:16.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:54:16.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'12057", 'reported_seq': 12139, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:13.886164+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:54:13.886164+0000', 'last_peered': '2024-09-16T04:54:13.886164+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:54:13.886164+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31461376, 'num_objects': 7681, 'num_object_clones': 0, 'num_object_copies': 15362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7681, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12057, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4340}, {'shards': '0,2', 'objects': 1152}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:16.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:54:16.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'12052", 'reported_seq': 12196, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:13.123780+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:54:13.123780+0000', 'last_peered': '2024-09-16T04:54:13.123780+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:54:13.123780+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31969304, 'num_objects': 7806, 'num_object_clones': 0, 'num_object_copies': 15612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7806, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8791, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1198}, {'shards': '2', 'objects': 4487}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:16.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:54:16.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'12341", 'reported_seq': 12479, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:13.825884+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:54:13.825884+0000', 'last_peered': '2024-09-16T04:54:13.825884+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:54:13.825884+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32653312, 'num_objects': 7972, 'num_object_clones': 0, 'num_object_copies': 15944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7972, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9334, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4516}, {'shards': '0,2', 'objects': 1274}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:16.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:54:16.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'12112", 'reported_seq': 12265, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:13.841501+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:54:13.841501+0000', 'last_peered': '2024-09-16T04:54:13.841501+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:54:13.841501+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31735808, 'num_objects': 7748, 'num_object_clones': 0, 'num_object_copies': 15496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7748, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8767, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4349}, {'shards': '0,2', 'objects': 1220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:16.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:54:16.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'12134", 'reported_seq': 12367, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:13.067724+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:54:13.067724+0000', 'last_peered': '2024-09-16T04:54:13.067724+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:54:13.067724+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32550912, 'num_objects': 7947, 'num_object_clones': 0, 'num_object_copies': 15894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7947, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8885, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1244}, {'shards': '2', 'objects': 4626}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:16.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:54:16.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'12343", 'reported_seq': 12478, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:13.881235+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:54:13.881235+0000', 'last_peered': '2024-09-16T04:54:13.881235+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:54:13.881235+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32256000, 'num_objects': 7875, 'num_object_clones': 0, 'num_object_copies': 15750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7875, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8998, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4410}, {'shards': '0,2', 'objects': 1235}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:16.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:54:16.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'12062", 'reported_seq': 12688, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:13.109431+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:54:13.109431+0000', 'last_peered': '2024-09-16T04:54:13.109431+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:54:13.109431+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32429266, 'num_objects': 7919, 'num_object_clones': 0, 'num_object_copies': 15838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1870, 'num_objects_misplaced': 1238, 'num_objects_unfound': 0, 'num_objects_dirty': 7919, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12034, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 192, 'num_bytes_recovered': 770048, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1238}, {'shards': '2', 'objects': 4811}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:16.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:54:16.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'12060", 'reported_seq': 12163, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:12.990287+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:54:12.990287+0000', 'last_peered': '2024-09-16T04:54:12.990287+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:54:12.990287+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32014953, 'num_objects': 7817, 'num_object_clones': 0, 'num_object_copies': 15634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7817, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12059, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1286}, {'shards': '2', 'objects': 4410}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:16.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:54:16.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'12464", 'reported_seq': 12572, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:13.121995+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:54:13.121995+0000', 'last_peered': '2024-09-16T04:54:13.121995+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:54:13.121995+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33048379, 'num_objects': 8071, 'num_object_clones': 0, 'num_object_copies': 16142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8071, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12461, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1320}, {'shards': '2', 'objects': 4557}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:16.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:54:16.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'12527", 'reported_seq': 12672, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:13.863904+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:54:13.863904+0000', 'last_peered': '2024-09-16T04:54:13.863904+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:54:13.863904+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32313961, 'num_objects': 7890, 'num_object_clones': 0, 'num_object_copies': 15780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7890, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9296, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4366}, {'shards': '0,2', 'objects': 1211}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:16.962 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:54:17.974 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:17.975 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32632/252968 objects degraded (12.900%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":405,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":126484,"data_bytes":518041866,"bytes_used":2138144768,"bytes_avail":287772147712,"bytes_total":289910292480,"degraded_objects":32632,"degraded_total":252968,"degraded_ratio":0.12899655292368994,"misplaced_objects":1238,"misplaced_total":252968,"misplaced_ratio":0.0048938996236678155,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":75200,"recovering_keys_per_sec":0,"num_objects_recovered":201,"num_bytes_recovered":815104,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":393},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:54:18.020 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:54:18.020 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:18.912 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:18.914 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:18.995 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-09-16T04:54:18.393191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":512585994,"num_objects":125152,"num_object_clones":0,"num_object_copies":250304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32697,"num_objects_misplaced":1095,"num_objects_unfound":0,"num_objects_dirty":125152,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161482,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6951,"num_bytes_recovered":28418067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160705,"ondisk_log_size":160705,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2076732,"kb_used_data":1119556,"kb_used_omap":12,"kb_used_meta":957107,"kb_avail":281038788,"statfs":{"total":289910292480,"available":287783718912,"internally_reserved":0,"allocated":1146425344,"data_stored":1145523774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":980077569},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,5,5,5,12,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22921216,"num_objects":-5596,"num_object_clones":0,"num_object_copies":-11192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1365,"num_objects_misplaced":1086,"num_objects_unfound":0,"num_objects_dirty":-5596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5596,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007303"},"pg_stats":[{"pgid":"2.f","version":"49'12057","reported_seq":12139,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.886164+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:13.886164+0000","last_peered":"2024-09-16T04:54:13.886164+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:13.886164+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31461376,"num_objects":7681,"num_object_clones":0,"num_object_copies":15362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12057,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4340},{"shards":"0,2","objects":1152}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12271","reported_seq":12392,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:18.118810+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:18.118810+0000","last_peered":"2024-09-16T04:54:18.118810+0000","last_clean":"2024-09-16T04:54:18.118810+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:18.118810+0000","last_undegraded":"2024-09-16T04:54:18.118810+0000","last_fullsized":"2024-09-16T04:54:18.118810+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31633408,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9240,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12196","reported_seq":12340,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.132611+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:18.132611+0000","last_peered":"2024-09-16T04:54:18.132611+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:18.132611+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31379480,"num_objects":7662,"num_object_clones":0,"num_object_copies":15324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8935,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1054},{"shards":"2","objects":4343}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12451","reported_seq":12526,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:18.017313+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:18.017313+0000","last_peered":"2024-09-16T04:54:18.017313+0000","last_clean":"2024-09-16T04:54:18.017313+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:18.017313+0000","last_undegraded":"2024-09-16T04:54:18.017313+0000","last_fullsized":"2024-09-16T04:54:18.017313+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32329728,"num_objects":7893,"num_object_clones":0,"num_object_copies":15786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12451,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12184","reported_seq":12408,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:13.770395+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:13.770395+0000","last_peered":"2024-09-16T04:54:13.770395+0000","last_clean":"2024-09-16T04:54:13.770395+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:13.770395+0000","last_undegraded":"2024-09-16T04:54:13.770395+0000","last_fullsized":"2024-09-16T04:54:13.770395+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32604160,"num_objects":7960,"num_object_clones":0,"num_object_copies":15920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'12341","reported_seq":12479,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.825884+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:13.825884+0000","last_peered":"2024-09-16T04:54:13.825884+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:13.825884+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32653312,"num_objects":7972,"num_object_clones":0,"num_object_copies":15944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9334,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4516},{"shards":"0,2","objects":1274}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12112","reported_seq":12265,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.841501+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:13.841501+0000","last_peered":"2024-09-16T04:54:13.841501+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:13.841501+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31735808,"num_objects":7748,"num_object_clones":0,"num_object_copies":15496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8767,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4349},{"shards":"0,2","objects":1220}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12102","reported_seq":12322,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:13.866537+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:13.866537+0000","last_peered":"2024-09-16T04:54:13.866537+0000","last_clean":"2024-09-16T04:54:13.866537+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:13.866537+0000","last_undegraded":"2024-09-16T04:54:13.866537+0000","last_fullsized":"2024-09-16T04:54:13.866537+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32415744,"num_objects":7914,"num_object_clones":0,"num_object_copies":15828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9172,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12290","reported_seq":12521,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.137166+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:18.137166+0000","last_peered":"2024-09-16T04:54:18.137166+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:18.137166+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31911936,"num_objects":7791,"num_object_clones":0,"num_object_copies":15582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9041,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1088},{"shards":"2","objects":4470}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'12343","reported_seq":12478,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.881235+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:13.881235+0000","last_peered":"2024-09-16T04:54:13.881235+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:13.881235+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32256000,"num_objects":7875,"num_object_clones":0,"num_object_copies":15750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8998,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4410},{"shards":"0,2","objects":1235}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556564+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556564+0000","last_peered":"2024-09-16T04:54:05.556564+0000","last_clean":"2024-09-16T04:54:05.556564+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556564+0000","last_undegraded":"2024-09-16T04:54:05.556564+0000","last_fullsized":"2024-09-16T04:54:05.556564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12204","reported_seq":12988,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.118026+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:18.118026+0000","last_peered":"2024-09-16T04:54:18.118026+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:18.118026+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31847634,"num_objects":7777,"num_object_clones":0,"num_object_copies":15554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1935,"num_objects_misplaced":1095,"num_objects_unfound":0,"num_objects_dirty":7777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12176,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1095},{"shards":"2","objects":4747}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12214","reported_seq":19075,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:17.952182+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:17.952182+0000","last_peered":"2024-09-16T04:54:17.952182+0000","last_clean":"2024-09-16T04:54:17.952182+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:17.952182+0000","last_undegraded":"2024-09-16T04:54:17.952182+0000","last_fullsized":"2024-09-16T04:54:17.952182+0000","mapping_epoch":46,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801344,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8937,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12222","reported_seq":12324,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.055237+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:18.055237+0000","last_peered":"2024-09-16T04:54:18.055237+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:18.055237+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31351401,"num_objects":7655,"num_object_clones":0,"num_object_copies":15310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12221,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1124},{"shards":"2","objects":4248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'12623","reported_seq":12731,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.137330+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:18.137330+0000","last_peered":"2024-09-16T04:54:18.137330+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:18.137330+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32397115,"num_objects":7912,"num_object_clones":0,"num_object_copies":15824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12620,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1161},{"shards":"2","objects":4398}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12265","reported_seq":19064,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:17.272747+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:17.272747+0000","last_peered":"2024-09-16T04:54:17.272747+0000","last_clean":"2024-09-16T04:54:17.272747+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:17.272747+0000","last_undegraded":"2024-09-16T04:54:17.272747+0000","last_fullsized":"2024-09-16T04:54:17.272747+0000","mapping_epoch":48,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32493568,"num_objects":7933,"num_object_clones":0,"num_object_copies":15866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9019,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'12527","reported_seq":12672,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:13.863904+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:13.863904+0000","last_peered":"2024-09-16T04:54:13.863904+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:13.863904+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313961,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9296,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4366},{"shards":"0,2","objects":1211}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":512585975,"num_objects":125150,"num_object_clones":0,"num_object_copies":250300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32697,"num_objects_misplaced":1095,"num_objects_unfound":0,"num_objects_dirty":125150,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161480,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6950,"num_bytes_recovered":28418048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1144745984,"data_stored":1144671739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736420,"kb_used_data":393744,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93635420,"statfs":{"total":96636764160,"available":95882670080,"internally_reserved":0,"allocated":403193856,"data_stored":402910530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593785,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609032,"kb_used_data":328824,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93762808,"statfs":{"total":96636764160,"available":96013115392,"internally_reserved":0,"allocated":336715776,"data_stored":336376786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":731280,"kb_used_data":396988,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93640560,"statfs":{"total":96636764160,"available":95887933440,"internally_reserved":0,"allocated":406515712,"data_stored":406236458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,0,2,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405925888,"data_stored":405922409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":336179200,"data_stored":336115985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402640896,"data_stored":402633345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:21.996 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:22.764 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:22.764 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:22.814 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-16T04:54:22.395445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":508043530,"num_objects":124043,"num_object_clones":0,"num_object_copies":248086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32697,"num_objects_misplaced":1095,"num_objects_unfound":0,"num_objects_dirty":124043,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162591,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6951,"num_bytes_recovered":28418067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2100600,"kb_used_data":1112900,"kb_used_omap":12,"kb_used_meta":987635,"kb_avail":281014920,"statfs":{"total":289910292480,"available":287759278080,"internally_reserved":0,"allocated":1139609600,"data_stored":1138708030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1011338241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,9,2,6,9,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19292160,"num_objects":-4710,"num_object_clones":0,"num_object_copies":-9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":126,"num_objects_misplaced":-286,"num_objects_unfound":0,"num_objects_dirty":-4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4710,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006061"},"pg_stats":[{"pgid":"2.f","version":"49'12201","reported_seq":12285,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.904364+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:18.904364+0000","last_peered":"2024-09-16T04:54:18.904364+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:18.904364+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30871552,"num_objects":7537,"num_object_clones":0,"num_object_copies":15074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12201,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4196},{"shards":"0,2","objects":1008}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12271","reported_seq":12392,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:18.118810+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:18.118810+0000","last_peered":"2024-09-16T04:54:18.118810+0000","last_clean":"2024-09-16T04:54:18.118810+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:18.118810+0000","last_undegraded":"2024-09-16T04:54:18.118810+0000","last_fullsized":"2024-09-16T04:54:18.118810+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31633408,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9240,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12196","reported_seq":12340,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.132611+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:18.132611+0000","last_peered":"2024-09-16T04:54:18.132611+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:18.132611+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31379480,"num_objects":7662,"num_object_clones":0,"num_object_copies":15324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8935,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1054},{"shards":"2","objects":4343}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12451","reported_seq":12526,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:18.017313+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:18.017313+0000","last_peered":"2024-09-16T04:54:18.017313+0000","last_clean":"2024-09-16T04:54:18.017313+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:18.017313+0000","last_undegraded":"2024-09-16T04:54:18.017313+0000","last_fullsized":"2024-09-16T04:54:18.017313+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32329728,"num_objects":7893,"num_object_clones":0,"num_object_copies":15786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12451,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12320","reported_seq":12544,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:18.851934+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:18.851934+0000","last_peered":"2024-09-16T04:54:18.851934+0000","last_clean":"2024-09-16T04:54:18.851934+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:18.851934+0000","last_undegraded":"2024-09-16T04:54:18.851934+0000","last_fullsized":"2024-09-16T04:54:18.851934+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32047104,"num_objects":7824,"num_object_clones":0,"num_object_copies":15648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9352,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'12482","reported_seq":12620,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.831253+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:18.831253+0000","last_peered":"2024-09-16T04:54:18.831253+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:18.831253+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32075776,"num_objects":7831,"num_object_clones":0,"num_object_copies":15662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9475,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4375},{"shards":"0,2","objects":1133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12255","reported_seq":12408,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.885369+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:18.885369+0000","last_peered":"2024-09-16T04:54:18.885369+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:18.885369+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31150080,"num_objects":7605,"num_object_clones":0,"num_object_copies":15210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8910,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4206},{"shards":"0,2","objects":1077}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12245","reported_seq":12466,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:18.913698+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:18.913698+0000","last_peered":"2024-09-16T04:54:18.913698+0000","last_clean":"2024-09-16T04:54:18.913698+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:18.913698+0000","last_undegraded":"2024-09-16T04:54:18.913698+0000","last_fullsized":"2024-09-16T04:54:18.913698+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31830016,"num_objects":7771,"num_object_clones":0,"num_object_copies":15542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9315,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12290","reported_seq":12521,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.137166+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:18.137166+0000","last_peered":"2024-09-16T04:54:18.137166+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:18.137166+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31911936,"num_objects":7791,"num_object_clones":0,"num_object_copies":15582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9041,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1088},{"shards":"2","objects":4470}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'12473","reported_seq":12608,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.912928+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:18.912928+0000","last_peered":"2024-09-16T04:54:18.912928+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:18.912928+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31723520,"num_objects":7745,"num_object_clones":0,"num_object_copies":15490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9128,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4280},{"shards":"0,2","objects":1105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556564+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556564+0000","last_peered":"2024-09-16T04:54:05.556564+0000","last_clean":"2024-09-16T04:54:05.556564+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556564+0000","last_undegraded":"2024-09-16T04:54:05.556564+0000","last_fullsized":"2024-09-16T04:54:05.556564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12204","reported_seq":12988,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.118026+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:18.118026+0000","last_peered":"2024-09-16T04:54:18.118026+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:18.118026+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31847634,"num_objects":7777,"num_object_clones":0,"num_object_copies":15554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1935,"num_objects_misplaced":1095,"num_objects_unfound":0,"num_objects_dirty":7777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12176,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1095},{"shards":"2","objects":4747}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12214","reported_seq":19075,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:17.952182+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:17.952182+0000","last_peered":"2024-09-16T04:54:17.952182+0000","last_clean":"2024-09-16T04:54:17.952182+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:17.952182+0000","last_undegraded":"2024-09-16T04:54:17.952182+0000","last_fullsized":"2024-09-16T04:54:17.952182+0000","mapping_epoch":46,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801344,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8937,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12222","reported_seq":12324,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.055237+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:18.055237+0000","last_peered":"2024-09-16T04:54:18.055237+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:18.055237+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31351401,"num_objects":7655,"num_object_clones":0,"num_object_copies":15310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12221,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1124},{"shards":"2","objects":4248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'12623","reported_seq":12731,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.137330+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:18.137330+0000","last_peered":"2024-09-16T04:54:18.137330+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:18.137330+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32397115,"num_objects":7912,"num_object_clones":0,"num_object_copies":15824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12620,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1161},{"shards":"2","objects":4398}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12417","reported_seq":19216,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:22.279041+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:22.279041+0000","last_peered":"2024-09-16T04:54:22.279041+0000","last_clean":"2024-09-16T04:54:22.279041+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:22.279041+0000","last_undegraded":"2024-09-16T04:54:22.279041+0000","last_fullsized":"2024-09-16T04:54:22.279041+0000","mapping_epoch":48,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31870976,"num_objects":7781,"num_object_clones":0,"num_object_copies":15562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9171,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'12647","reported_seq":12792,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.877489+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:18.877489+0000","last_peered":"2024-09-16T04:54:18.877489+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:18.877489+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31822441,"num_objects":7770,"num_object_clones":0,"num_object_copies":15540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9416,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4246},{"shards":"0,2","objects":1091}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":508043511,"num_objects":124041,"num_object_clones":0,"num_object_copies":248082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32697,"num_objects_misplaced":1095,"num_objects_unfound":0,"num_objects_dirty":124041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162589,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6950,"num_bytes_recovered":28418048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1137872896,"data_stored":1137798651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736420,"kb_used_data":393744,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93635420,"statfs":{"total":96636764160,"available":95882670080,"internally_reserved":0,"allocated":403193856,"data_stored":402910530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593786,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634200,"kb_used_data":327624,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93737640,"statfs":{"total":96636764160,"available":95987343360,"internally_reserved":0,"allocated":335486976,"data_stored":335147986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,1,2,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729980,"kb_used_data":391532,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93641860,"statfs":{"total":96636764160,"available":95889264640,"internally_reserved":0,"allocated":400928768,"data_stored":400649514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,0,3,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":400302080,"data_stored":400298601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334929920,"data_stored":334866705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402640896,"data_stored":402633345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:22.815 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:23.609 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:23.650 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-16T04:54:22.395445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":508043530,"num_objects":124043,"num_object_clones":0,"num_object_copies":248086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32697,"num_objects_misplaced":1095,"num_objects_unfound":0,"num_objects_dirty":124043,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162591,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6951,"num_bytes_recovered":28418067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2100600,"kb_used_data":1112900,"kb_used_omap":12,"kb_used_meta":987635,"kb_avail":281014920,"statfs":{"total":289910292480,"available":287759278080,"internally_reserved":0,"allocated":1139609600,"data_stored":1138708030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1011338241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,9,2,6,9,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19292160,"num_objects":-4710,"num_object_clones":0,"num_object_copies":-9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":126,"num_objects_misplaced":-286,"num_objects_unfound":0,"num_objects_dirty":-4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4710,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006061"},"pg_stats":[{"pgid":"2.f","version":"49'12201","reported_seq":12285,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.904364+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:18.904364+0000","last_peered":"2024-09-16T04:54:18.904364+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:18.904364+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30871552,"num_objects":7537,"num_object_clones":0,"num_object_copies":15074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12201,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4196},{"shards":"0,2","objects":1008}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12271","reported_seq":12392,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:18.118810+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:18.118810+0000","last_peered":"2024-09-16T04:54:18.118810+0000","last_clean":"2024-09-16T04:54:18.118810+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:18.118810+0000","last_undegraded":"2024-09-16T04:54:18.118810+0000","last_fullsized":"2024-09-16T04:54:18.118810+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31633408,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9240,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12196","reported_seq":12340,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.132611+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:18.132611+0000","last_peered":"2024-09-16T04:54:18.132611+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:18.132611+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31379480,"num_objects":7662,"num_object_clones":0,"num_object_copies":15324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8935,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1054},{"shards":"2","objects":4343}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12451","reported_seq":12526,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:18.017313+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:18.017313+0000","last_peered":"2024-09-16T04:54:18.017313+0000","last_clean":"2024-09-16T04:54:18.017313+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:18.017313+0000","last_undegraded":"2024-09-16T04:54:18.017313+0000","last_fullsized":"2024-09-16T04:54:18.017313+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32329728,"num_objects":7893,"num_object_clones":0,"num_object_copies":15786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12451,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12320","reported_seq":12544,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:18.851934+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:18.851934+0000","last_peered":"2024-09-16T04:54:18.851934+0000","last_clean":"2024-09-16T04:54:18.851934+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:18.851934+0000","last_undegraded":"2024-09-16T04:54:18.851934+0000","last_fullsized":"2024-09-16T04:54:18.851934+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32047104,"num_objects":7824,"num_object_clones":0,"num_object_copies":15648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9352,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'12482","reported_seq":12620,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.831253+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:18.831253+0000","last_peered":"2024-09-16T04:54:18.831253+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:18.831253+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32075776,"num_objects":7831,"num_object_clones":0,"num_object_copies":15662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9475,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4375},{"shards":"0,2","objects":1133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12255","reported_seq":12408,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.885369+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:18.885369+0000","last_peered":"2024-09-16T04:54:18.885369+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:18.885369+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31150080,"num_objects":7605,"num_object_clones":0,"num_object_copies":15210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8910,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4206},{"shards":"0,2","objects":1077}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12245","reported_seq":12466,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:18.913698+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:18.913698+0000","last_peered":"2024-09-16T04:54:18.913698+0000","last_clean":"2024-09-16T04:54:18.913698+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:18.913698+0000","last_undegraded":"2024-09-16T04:54:18.913698+0000","last_fullsized":"2024-09-16T04:54:18.913698+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31830016,"num_objects":7771,"num_object_clones":0,"num_object_copies":15542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9315,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12290","reported_seq":12521,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.137166+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:18.137166+0000","last_peered":"2024-09-16T04:54:18.137166+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:18.137166+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31911936,"num_objects":7791,"num_object_clones":0,"num_object_copies":15582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9041,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1088},{"shards":"2","objects":4470}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'12473","reported_seq":12608,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.912928+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:18.912928+0000","last_peered":"2024-09-16T04:54:18.912928+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:18.912928+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31723520,"num_objects":7745,"num_object_clones":0,"num_object_copies":15490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9128,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4280},{"shards":"0,2","objects":1105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556564+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556564+0000","last_peered":"2024-09-16T04:54:05.556564+0000","last_clean":"2024-09-16T04:54:05.556564+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556564+0000","last_undegraded":"2024-09-16T04:54:05.556564+0000","last_fullsized":"2024-09-16T04:54:05.556564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12204","reported_seq":12988,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.118026+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:18.118026+0000","last_peered":"2024-09-16T04:54:18.118026+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:18.118026+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31847634,"num_objects":7777,"num_object_clones":0,"num_object_copies":15554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1935,"num_objects_misplaced":1095,"num_objects_unfound":0,"num_objects_dirty":7777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12176,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1095},{"shards":"2","objects":4747}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12214","reported_seq":19075,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:17.952182+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:17.952182+0000","last_peered":"2024-09-16T04:54:17.952182+0000","last_clean":"2024-09-16T04:54:17.952182+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:17.952182+0000","last_undegraded":"2024-09-16T04:54:17.952182+0000","last_fullsized":"2024-09-16T04:54:17.952182+0000","mapping_epoch":46,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801344,"num_objects":7764,"num_object_clones":0,"num_object_copies":15528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8937,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12222","reported_seq":12324,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.055237+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:18.055237+0000","last_peered":"2024-09-16T04:54:18.055237+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:18.055237+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31351401,"num_objects":7655,"num_object_clones":0,"num_object_copies":15310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12221,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1124},{"shards":"2","objects":4248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'12623","reported_seq":12731,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.137330+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:18.137330+0000","last_peered":"2024-09-16T04:54:18.137330+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:18.137330+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32397115,"num_objects":7912,"num_object_clones":0,"num_object_copies":15824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12620,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1161},{"shards":"2","objects":4398}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12417","reported_seq":19216,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:22.279041+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:22.279041+0000","last_peered":"2024-09-16T04:54:22.279041+0000","last_clean":"2024-09-16T04:54:22.279041+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:22.279041+0000","last_undegraded":"2024-09-16T04:54:22.279041+0000","last_fullsized":"2024-09-16T04:54:22.279041+0000","mapping_epoch":48,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31870976,"num_objects":7781,"num_object_clones":0,"num_object_copies":15562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9171,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'12647","reported_seq":12792,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:18.877489+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:18.877489+0000","last_peered":"2024-09-16T04:54:18.877489+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:18.877489+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31822441,"num_objects":7770,"num_object_clones":0,"num_object_copies":15540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9416,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4246},{"shards":"0,2","objects":1091}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":508043511,"num_objects":124041,"num_object_clones":0,"num_object_copies":248082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32697,"num_objects_misplaced":1095,"num_objects_unfound":0,"num_objects_dirty":124041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":162589,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6950,"num_bytes_recovered":28418048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1137872896,"data_stored":1137798651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736420,"kb_used_data":393744,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93635420,"statfs":{"total":96636764160,"available":95882670080,"internally_reserved":0,"allocated":403193856,"data_stored":402910530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593786,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634200,"kb_used_data":327624,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93737640,"statfs":{"total":96636764160,"available":95987343360,"internally_reserved":0,"allocated":335486976,"data_stored":335147986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,1,2,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729980,"kb_used_data":391532,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93641860,"statfs":{"total":96636764160,"available":95889264640,"internally_reserved":0,"allocated":400928768,"data_stored":400649514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,0,3,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":400302080,"data_stored":400298601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334929920,"data_stored":334866705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402640896,"data_stored":402633345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:23.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:54:23.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'12201", 'reported_seq': 12285, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:18.904364+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:54:18.904364+0000', 'last_peered': '2024-09-16T04:54:18.904364+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:54:18.904364+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30871552, 'num_objects': 7537, 'num_object_clones': 0, 'num_object_copies': 15074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7537, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12201, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4196}, {'shards': '0,2', 'objects': 1008}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:23.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:54:23.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'12196", 'reported_seq': 12340, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:18.132611+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:54:18.132611+0000', 'last_peered': '2024-09-16T04:54:18.132611+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:54:18.132611+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31379480, 'num_objects': 7662, 'num_object_clones': 0, 'num_object_copies': 15324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7662, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8935, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1054}, {'shards': '2', 'objects': 4343}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:23.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:54:23.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'12482", 'reported_seq': 12620, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:18.831253+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:54:18.831253+0000', 'last_peered': '2024-09-16T04:54:18.831253+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:54:18.831253+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32075776, 'num_objects': 7831, 'num_object_clones': 0, 'num_object_copies': 15662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7831, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9475, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4375}, {'shards': '0,2', 'objects': 1133}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:23.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:54:23.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'12255", 'reported_seq': 12408, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:18.885369+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:54:18.885369+0000', 'last_peered': '2024-09-16T04:54:18.885369+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:54:18.885369+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31150080, 'num_objects': 7605, 'num_object_clones': 0, 'num_object_copies': 15210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7605, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8910, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4206}, {'shards': '0,2', 'objects': 1077}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:23.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:54:23.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'12290", 'reported_seq': 12521, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:18.137166+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:54:18.137166+0000', 'last_peered': '2024-09-16T04:54:18.137166+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:54:18.137166+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31911936, 'num_objects': 7791, 'num_object_clones': 0, 'num_object_copies': 15582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7791, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9041, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1088}, {'shards': '2', 'objects': 4470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:23.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:54:23.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'12473", 'reported_seq': 12608, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:18.912928+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:54:18.912928+0000', 'last_peered': '2024-09-16T04:54:18.912928+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:54:18.912928+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31723520, 'num_objects': 7745, 'num_object_clones': 0, 'num_object_copies': 15490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7745, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9128, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4280}, {'shards': '0,2', 'objects': 1105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:23.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:54:23.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'12204", 'reported_seq': 12988, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:18.118026+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:54:18.118026+0000', 'last_peered': '2024-09-16T04:54:18.118026+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:54:18.118026+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31847634, 'num_objects': 7777, 'num_object_clones': 0, 'num_object_copies': 15554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1935, 'num_objects_misplaced': 1095, 'num_objects_unfound': 0, 'num_objects_dirty': 7777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12176, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 270, 'num_bytes_recovered': 1089536, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1095}, {'shards': '2', 'objects': 4747}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:23.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:54:23.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'12222", 'reported_seq': 12324, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:18.055237+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:54:18.055237+0000', 'last_peered': '2024-09-16T04:54:18.055237+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:54:18.055237+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31351401, 'num_objects': 7655, 'num_object_clones': 0, 'num_object_copies': 15310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7655, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12221, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1124}, {'shards': '2', 'objects': 4248}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:23.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:54:23.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'12623", 'reported_seq': 12731, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:18.137330+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:54:18.137330+0000', 'last_peered': '2024-09-16T04:54:18.137330+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:54:18.137330+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32397115, 'num_objects': 7912, 'num_object_clones': 0, 'num_object_copies': 15824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7912, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12620, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1161}, {'shards': '2', 'objects': 4398}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:23.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:54:23.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'12647", 'reported_seq': 12792, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:18.877489+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:54:18.877489+0000', 'last_peered': '2024-09-16T04:54:18.877489+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:54:18.877489+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31822441, 'num_objects': 7770, 'num_object_clones': 0, 'num_object_copies': 15540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9416, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4246}, {'shards': '0,2', 'objects': 1091}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:23.657 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:54:24.620 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:24.621 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32740/243674 objects degraded (13.436%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":412,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":121837,"data_bytes":499007754,"bytes_used":2147684352,"bytes_avail":287762608128,"bytes_total":289910292480,"degraded_objects":32740,"degraded_total":243674,"degraded_ratio":0.13435984142748097,"misplaced_objects":953,"misplaced_total":243674,"misplaced_ratio":0.0039109630079532491,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":88295,"recovering_keys_per_sec":0,"num_objects_recovered":259,"num_bytes_recovered":1060864,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":563},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:54:24.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:54:24.663 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:25.550 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:25.551 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:25.591 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-16T04:54:24.404790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":499007754,"num_objects":121837,"num_object_clones":0,"num_object_copies":243674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32740,"num_objects_misplaced":953,"num_objects_unfound":0,"num_objects_dirty":121837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164797,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7050,"num_bytes_recovered":28823571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160720,"ondisk_log_size":160720,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2097348,"kb_used_data":1101328,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":281018172,"statfs":{"total":289910292480,"available":287762608128,"internally_reserved":0,"allocated":1127759872,"data_stored":1126858302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1019857921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,4,8,5,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27729920,"num_objects":-6770,"num_object_clones":0,"num_object_copies":-13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":169,"num_objects_misplaced":-428,"num_objects_unfound":0,"num_objects_dirty":-6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6770,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":1060864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.014930"},"pg_stats":[{"pgid":"2.f","version":"49'12322","reported_seq":12406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.857275+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:23.857275+0000","last_peered":"2024-09-16T04:54:23.857275+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:23.857275+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30375936,"num_objects":7416,"num_object_clones":0,"num_object_copies":14832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12322,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4075},{"shards":"0,2","objects":887}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12410","reported_seq":12531,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:23.026434+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:23.026434+0000","last_peered":"2024-09-16T04:54:23.026434+0000","last_clean":"2024-09-16T04:54:23.026434+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:23.026434+0000","last_undegraded":"2024-09-16T04:54:23.026434+0000","last_fullsized":"2024-09-16T04:54:23.026434+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31064064,"num_objects":7584,"num_object_clones":0,"num_object_copies":15168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9379,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12332","reported_seq":12476,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:22.966764+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:22.966764+0000","last_peered":"2024-09-16T04:54:22.966764+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:22.966764+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30822424,"num_objects":7526,"num_object_clones":0,"num_object_copies":15052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9071,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":918},{"shards":"2","objects":4207}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12590","reported_seq":12665,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:23.181401+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:23.181401+0000","last_peered":"2024-09-16T04:54:23.181401+0000","last_clean":"2024-09-16T04:54:23.181401+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:23.181401+0000","last_undegraded":"2024-09-16T04:54:23.181401+0000","last_fullsized":"2024-09-16T04:54:23.181401+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31760384,"num_objects":7754,"num_object_clones":0,"num_object_copies":15508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12590,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12471","reported_seq":12694,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:23.885214+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:23.885214+0000","last_peered":"2024-09-16T04:54:23.885214+0000","last_clean":"2024-09-16T04:54:23.885214+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:23.885214+0000","last_undegraded":"2024-09-16T04:54:23.885214+0000","last_fullsized":"2024-09-16T04:54:23.885214+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31428608,"num_objects":7673,"num_object_clones":0,"num_object_copies":15346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9503,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'12654","reported_seq":12792,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.836949+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:23.836949+0000","last_peered":"2024-09-16T04:54:23.836949+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:23.836949+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31371264,"num_objects":7659,"num_object_clones":0,"num_object_copies":15318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9647,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4203},{"shards":"0,2","objects":961}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12395","reported_seq":12548,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.861939+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:23.861939+0000","last_peered":"2024-09-16T04:54:23.861939+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:23.861939+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30576640,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9050,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4066},{"shards":"0,2","objects":937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12408","reported_seq":12629,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:23.909602+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:23.909602+0000","last_peered":"2024-09-16T04:54:23.909602+0000","last_clean":"2024-09-16T04:54:23.909602+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:23.909602+0000","last_undegraded":"2024-09-16T04:54:23.909602+0000","last_fullsized":"2024-09-16T04:54:23.909602+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162368,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9478,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12421","reported_seq":12654,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.207142+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:23.207142+0000","last_peered":"2024-09-16T04:54:23.207142+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:23.207142+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31375360,"num_objects":7660,"num_object_clones":0,"num_object_copies":15320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9172,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":957},{"shards":"2","objects":4339}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'12617","reported_seq":12752,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.810012+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:23.810012+0000","last_peered":"2024-09-16T04:54:23.810012+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:23.810012+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31133696,"num_objects":7601,"num_object_clones":0,"num_object_copies":15202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9272,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4136},{"shards":"0,2","objects":961}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556564+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556564+0000","last_peered":"2024-09-16T04:54:05.556564+0000","last_clean":"2024-09-16T04:54:05.556564+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556564+0000","last_undegraded":"2024-09-16T04:54:05.556564+0000","last_fullsized":"2024-09-16T04:54:05.556564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12347","reported_seq":13329,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.200919+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:23.200919+0000","last_peered":"2024-09-16T04:54:23.200919+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:23.200919+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31261906,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1978,"num_objects_misplaced":953,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12319,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":1495040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":953},{"shards":"2","objects":4703}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12368","reported_seq":19229,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:23.075866+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:23.075866+0000","last_peered":"2024-09-16T04:54:23.075866+0000","last_clean":"2024-09-16T04:54:23.075866+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:23.075866+0000","last_undegraded":"2024-09-16T04:54:23.075866+0000","last_fullsized":"2024-09-16T04:54:23.075866+0000","mapping_epoch":46,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31170560,"num_objects":7610,"num_object_clones":0,"num_object_copies":15220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9091,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12371","reported_seq":12474,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.127290+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:23.127290+0000","last_peered":"2024-09-16T04:54:23.127290+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:23.127290+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30741097,"num_objects":7506,"num_object_clones":0,"num_object_copies":15012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12370,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":975},{"shards":"2","objects":4099}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'12790","reported_seq":12898,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.148874+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:23.148874+0000","last_peered":"2024-09-16T04:54:23.148874+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:23.148874+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31713083,"num_objects":7745,"num_object_clones":0,"num_object_copies":15490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12787,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":994},{"shards":"2","objects":4231}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12417","reported_seq":19216,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:22.279041+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:22.279041+0000","last_peered":"2024-09-16T04:54:22.279041+0000","last_clean":"2024-09-16T04:54:22.279041+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:22.279041+0000","last_undegraded":"2024-09-16T04:54:22.279041+0000","last_fullsized":"2024-09-16T04:54:22.279041+0000","mapping_epoch":48,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31870976,"num_objects":7781,"num_object_clones":0,"num_object_copies":15562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9171,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'12804","reported_seq":12949,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.900424+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:23.900424+0000","last_peered":"2024-09-16T04:54:23.900424+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:23.900424+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31179369,"num_objects":7613,"num_object_clones":0,"num_object_copies":15226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9573,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4089},{"shards":"0,2","objects":934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":499007735,"num_objects":121835,"num_object_clones":0,"num_object_copies":243670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32740,"num_objects_misplaced":953,"num_objects_unfound":0,"num_objects_dirty":121835,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164795,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7049,"num_bytes_recovered":28823552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1125945344,"data_stored":1125871099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160717,"ondisk_log_size":160717,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":734840,"kb_used_data":388008,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93637000,"statfs":{"total":96636764160,"available":95884288000,"internally_reserved":0,"allocated":397320192,"data_stored":397036866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593786,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634200,"kb_used_data":327624,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93737640,"statfs":{"total":96636764160,"available":95987343360,"internally_reserved":0,"allocated":335486976,"data_stored":335147986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,1,2,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728308,"kb_used_data":385696,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93643532,"statfs":{"total":96636764160,"available":95890976768,"internally_reserved":0,"allocated":394952704,"data_stored":394673450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,1,0,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":394326016,"data_stored":394322537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":334929920,"data_stored":334866705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396689408,"data_stored":396681857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:28.593 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:29.421 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:29.421 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:29.465 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-16T04:54:28.405889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":493850890,"num_objects":120578,"num_object_clones":0,"num_object_copies":241156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32800,"num_objects_misplaced":807,"num_objects_unfound":0,"num_objects_dirty":120578,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":166056,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7136,"num_bytes_recovered":29175827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160779,"ondisk_log_size":160779,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2082332,"kb_used_data":1094624,"kb_used_omap":12,"kb_used_meta":987635,"kb_avail":281033188,"statfs":{"total":289910292480,"available":287777984512,"internally_reserved":0,"allocated":1120894976,"data_stored":1119993406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1011338241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,8,10,5,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24190976,"num_objects":-5906,"num_object_clones":0,"num_object_copies":-11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":168,"num_objects_misplaced":-431,"num_objects_unfound":0,"num_objects_dirty":-5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5906,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":1077248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013114"},"pg_stats":[{"pgid":"2.f","version":"49'12322","reported_seq":12406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.857275+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:23.857275+0000","last_peered":"2024-09-16T04:54:23.857275+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:23.857275+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30375936,"num_objects":7416,"num_object_clones":0,"num_object_copies":14832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12322,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4075},{"shards":"0,2","objects":887}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12551","reported_seq":12672,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:28.157955+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:28.157955+0000","last_peered":"2024-09-16T04:54:28.157955+0000","last_clean":"2024-09-16T04:54:28.157955+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:28.157955+0000","last_undegraded":"2024-09-16T04:54:28.157955+0000","last_fullsized":"2024-09-16T04:54:28.157955+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30486528,"num_objects":7443,"num_object_clones":0,"num_object_copies":14886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9520,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12466","reported_seq":12609,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.208807+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:28.208807+0000","last_peered":"2024-09-16T04:54:28.208807+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:28.208807+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30273560,"num_objects":7392,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9205,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":784},{"shards":"2","objects":4073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12731","reported_seq":12806,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:28.200413+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:28.200413+0000","last_peered":"2024-09-16T04:54:28.200413+0000","last_clean":"2024-09-16T04:54:28.200413+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:28.200413+0000","last_undegraded":"2024-09-16T04:54:28.200413+0000","last_fullsized":"2024-09-16T04:54:28.200413+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31182848,"num_objects":7613,"num_object_clones":0,"num_object_copies":15226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12731,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12471","reported_seq":12694,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:23.885214+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:23.885214+0000","last_peered":"2024-09-16T04:54:23.885214+0000","last_clean":"2024-09-16T04:54:23.885214+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:23.885214+0000","last_undegraded":"2024-09-16T04:54:23.885214+0000","last_fullsized":"2024-09-16T04:54:23.885214+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31428608,"num_objects":7673,"num_object_clones":0,"num_object_copies":15346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9503,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'12654","reported_seq":12792,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.836949+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:23.836949+0000","last_peered":"2024-09-16T04:54:23.836949+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:23.836949+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31371264,"num_objects":7659,"num_object_clones":0,"num_object_copies":15318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9647,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4203},{"shards":"0,2","objects":961}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12395","reported_seq":12548,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.861939+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:23.861939+0000","last_peered":"2024-09-16T04:54:23.861939+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:23.861939+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30576640,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9050,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4066},{"shards":"0,2","objects":937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12408","reported_seq":12629,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:23.909602+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:23.909602+0000","last_peered":"2024-09-16T04:54:23.909602+0000","last_clean":"2024-09-16T04:54:23.909602+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:23.909602+0000","last_undegraded":"2024-09-16T04:54:23.909602+0000","last_fullsized":"2024-09-16T04:54:23.909602+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162368,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9478,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12565","reported_seq":12798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.141818+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:28.141818+0000","last_peered":"2024-09-16T04:54:28.141818+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:28.141818+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30785536,"num_objects":7516,"num_object_clones":0,"num_object_copies":15032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9316,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":813},{"shards":"2","objects":4195}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'12617","reported_seq":12752,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.810012+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:23.810012+0000","last_peered":"2024-09-16T04:54:23.810012+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:23.810012+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31133696,"num_objects":7601,"num_object_clones":0,"num_object_copies":15202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9272,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4136},{"shards":"0,2","objects":961}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556564+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556564+0000","last_peered":"2024-09-16T04:54:05.556564+0000","last_clean":"2024-09-16T04:54:05.556564+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556564+0000","last_undegraded":"2024-09-16T04:54:05.556564+0000","last_fullsized":"2024-09-16T04:54:05.556564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12493","reported_seq":13646,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.188493+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:28.188493+0000","last_peered":"2024-09-16T04:54:28.188493+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:28.188493+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30663890,"num_objects":7488,"num_object_clones":0,"num_object_copies":14976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2038,"num_objects_misplaced":807,"num_objects_unfound":0,"num_objects_dirty":7488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12465,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":455,"num_bytes_recovered":1847296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":807},{"shards":"2","objects":4643}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12512","reported_seq":19373,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:28.080837+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:28.080837+0000","last_peered":"2024-09-16T04:54:28.080837+0000","last_clean":"2024-09-16T04:54:28.080837+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:28.080837+0000","last_undegraded":"2024-09-16T04:54:28.080837+0000","last_fullsized":"2024-09-16T04:54:28.080837+0000","mapping_epoch":46,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30580736,"num_objects":7466,"num_object_clones":0,"num_object_copies":14932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9235,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12500","reported_seq":12603,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.099859+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:28.099859+0000","last_peered":"2024-09-16T04:54:28.099859+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:28.099859+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30212713,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12499,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":846},{"shards":"2","objects":3970}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'12943","reported_seq":13051,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.229358+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:28.229358+0000","last_peered":"2024-09-16T04:54:28.229358+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:28.229358+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31086395,"num_objects":7592,"num_object_clones":0,"num_object_copies":15184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12940,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":841},{"shards":"2","objects":4078}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12544","reported_seq":19343,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:27.330190+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:27.330190+0000","last_peered":"2024-09-16T04:54:27.330190+0000","last_clean":"2024-09-16T04:54:27.330190+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:27.330190+0000","last_undegraded":"2024-09-16T04:54:27.330190+0000","last_fullsized":"2024-09-16T04:54:27.330190+0000","mapping_epoch":48,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31350784,"num_objects":7654,"num_object_clones":0,"num_object_copies":15308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9298,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'12804","reported_seq":12949,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.900424+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:23.900424+0000","last_peered":"2024-09-16T04:54:23.900424+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:23.900424+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31179369,"num_objects":7613,"num_object_clones":0,"num_object_copies":15226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9573,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4089},{"shards":"0,2","objects":934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":493850871,"num_objects":120576,"num_object_clones":0,"num_object_copies":241152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32800,"num_objects_misplaced":807,"num_objects_unfound":0,"num_objects_dirty":120576,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":166054,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7135,"num_bytes_recovered":29175808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1119051776,"data_stored":1118977531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716688,"kb_used_data":382332,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93655152,"statfs":{"total":96636764160,"available":95902875648,"internally_reserved":0,"allocated":391507968,"data_stored":391224642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,3,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593787,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637336,"kb_used_data":326596,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93734504,"statfs":{"total":96636764160,"available":95984132096,"internally_reserved":0,"allocated":334434304,"data_stored":334095314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728308,"kb_used_data":385696,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93643532,"statfs":{"total":96636764160,"available":95890976768,"internally_reserved":0,"allocated":394952704,"data_stored":394673450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,1,0,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":394326016,"data_stored":394322537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333848576,"data_stored":333785361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390877184,"data_stored":390869633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:29.466 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:30.218 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:30.218 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:30.281 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-16T04:54:28.405889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":493850890,"num_objects":120578,"num_object_clones":0,"num_object_copies":241156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32800,"num_objects_misplaced":807,"num_objects_unfound":0,"num_objects_dirty":120578,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":166056,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7136,"num_bytes_recovered":29175827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160779,"ondisk_log_size":160779,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2082332,"kb_used_data":1094624,"kb_used_omap":12,"kb_used_meta":987635,"kb_avail":281033188,"statfs":{"total":289910292480,"available":287777984512,"internally_reserved":0,"allocated":1120894976,"data_stored":1119993406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1011338241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,8,10,5,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24190976,"num_objects":-5906,"num_object_clones":0,"num_object_copies":-11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":168,"num_objects_misplaced":-431,"num_objects_unfound":0,"num_objects_dirty":-5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5906,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":1077248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013114"},"pg_stats":[{"pgid":"2.f","version":"49'12322","reported_seq":12406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.857275+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:23.857275+0000","last_peered":"2024-09-16T04:54:23.857275+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:23.857275+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30375936,"num_objects":7416,"num_object_clones":0,"num_object_copies":14832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12322,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4075},{"shards":"0,2","objects":887}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12551","reported_seq":12672,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:28.157955+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:28.157955+0000","last_peered":"2024-09-16T04:54:28.157955+0000","last_clean":"2024-09-16T04:54:28.157955+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:28.157955+0000","last_undegraded":"2024-09-16T04:54:28.157955+0000","last_fullsized":"2024-09-16T04:54:28.157955+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30486528,"num_objects":7443,"num_object_clones":0,"num_object_copies":14886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9520,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12466","reported_seq":12609,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.208807+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:28.208807+0000","last_peered":"2024-09-16T04:54:28.208807+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:28.208807+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30273560,"num_objects":7392,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9205,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":784},{"shards":"2","objects":4073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12731","reported_seq":12806,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:28.200413+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:28.200413+0000","last_peered":"2024-09-16T04:54:28.200413+0000","last_clean":"2024-09-16T04:54:28.200413+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:28.200413+0000","last_undegraded":"2024-09-16T04:54:28.200413+0000","last_fullsized":"2024-09-16T04:54:28.200413+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31182848,"num_objects":7613,"num_object_clones":0,"num_object_copies":15226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12731,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12471","reported_seq":12694,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:23.885214+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:23.885214+0000","last_peered":"2024-09-16T04:54:23.885214+0000","last_clean":"2024-09-16T04:54:23.885214+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:23.885214+0000","last_undegraded":"2024-09-16T04:54:23.885214+0000","last_fullsized":"2024-09-16T04:54:23.885214+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31428608,"num_objects":7673,"num_object_clones":0,"num_object_copies":15346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9503,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'12654","reported_seq":12792,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.836949+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:23.836949+0000","last_peered":"2024-09-16T04:54:23.836949+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:23.836949+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31371264,"num_objects":7659,"num_object_clones":0,"num_object_copies":15318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9647,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4203},{"shards":"0,2","objects":961}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12395","reported_seq":12548,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.861939+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:23.861939+0000","last_peered":"2024-09-16T04:54:23.861939+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:23.861939+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30576640,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9050,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4066},{"shards":"0,2","objects":937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12408","reported_seq":12629,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:23.909602+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:23.909602+0000","last_peered":"2024-09-16T04:54:23.909602+0000","last_clean":"2024-09-16T04:54:23.909602+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:23.909602+0000","last_undegraded":"2024-09-16T04:54:23.909602+0000","last_fullsized":"2024-09-16T04:54:23.909602+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31162368,"num_objects":7608,"num_object_clones":0,"num_object_copies":15216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9478,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12565","reported_seq":12798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.141818+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:28.141818+0000","last_peered":"2024-09-16T04:54:28.141818+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:28.141818+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30785536,"num_objects":7516,"num_object_clones":0,"num_object_copies":15032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9316,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":813},{"shards":"2","objects":4195}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'12617","reported_seq":12752,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.810012+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:23.810012+0000","last_peered":"2024-09-16T04:54:23.810012+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:23.810012+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31133696,"num_objects":7601,"num_object_clones":0,"num_object_copies":15202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9272,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4136},{"shards":"0,2","objects":961}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556564+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556564+0000","last_peered":"2024-09-16T04:54:05.556564+0000","last_clean":"2024-09-16T04:54:05.556564+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556564+0000","last_undegraded":"2024-09-16T04:54:05.556564+0000","last_fullsized":"2024-09-16T04:54:05.556564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12493","reported_seq":13646,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.188493+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:28.188493+0000","last_peered":"2024-09-16T04:54:28.188493+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:28.188493+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30663890,"num_objects":7488,"num_object_clones":0,"num_object_copies":14976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2038,"num_objects_misplaced":807,"num_objects_unfound":0,"num_objects_dirty":7488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12465,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":455,"num_bytes_recovered":1847296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":807},{"shards":"2","objects":4643}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12512","reported_seq":19373,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:28.080837+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:28.080837+0000","last_peered":"2024-09-16T04:54:28.080837+0000","last_clean":"2024-09-16T04:54:28.080837+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:28.080837+0000","last_undegraded":"2024-09-16T04:54:28.080837+0000","last_fullsized":"2024-09-16T04:54:28.080837+0000","mapping_epoch":46,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30580736,"num_objects":7466,"num_object_clones":0,"num_object_copies":14932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9235,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12500","reported_seq":12603,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.099859+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:28.099859+0000","last_peered":"2024-09-16T04:54:28.099859+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:28.099859+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30212713,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12499,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":846},{"shards":"2","objects":3970}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'12943","reported_seq":13051,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.229358+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:28.229358+0000","last_peered":"2024-09-16T04:54:28.229358+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:28.229358+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31086395,"num_objects":7592,"num_object_clones":0,"num_object_copies":15184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12940,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":841},{"shards":"2","objects":4078}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12544","reported_seq":19343,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:27.330190+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:27.330190+0000","last_peered":"2024-09-16T04:54:27.330190+0000","last_clean":"2024-09-16T04:54:27.330190+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:27.330190+0000","last_undegraded":"2024-09-16T04:54:27.330190+0000","last_fullsized":"2024-09-16T04:54:27.330190+0000","mapping_epoch":48,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31350784,"num_objects":7654,"num_object_clones":0,"num_object_copies":15308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9298,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'12804","reported_seq":12949,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:23.900424+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:23.900424+0000","last_peered":"2024-09-16T04:54:23.900424+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:23.900424+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31179369,"num_objects":7613,"num_object_clones":0,"num_object_copies":15226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9573,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4089},{"shards":"0,2","objects":934}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":493850871,"num_objects":120576,"num_object_clones":0,"num_object_copies":241152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32800,"num_objects_misplaced":807,"num_objects_unfound":0,"num_objects_dirty":120576,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":166054,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7135,"num_bytes_recovered":29175808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1119051776,"data_stored":1118977531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160776,"ondisk_log_size":160776,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716688,"kb_used_data":382332,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93655152,"statfs":{"total":96636764160,"available":95902875648,"internally_reserved":0,"allocated":391507968,"data_stored":391224642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,3,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593787,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637336,"kb_used_data":326596,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93734504,"statfs":{"total":96636764160,"available":95984132096,"internally_reserved":0,"allocated":334434304,"data_stored":334095314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728308,"kb_used_data":385696,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93643532,"statfs":{"total":96636764160,"available":95890976768,"internally_reserved":0,"allocated":394952704,"data_stored":394673450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,1,0,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":394326016,"data_stored":394322537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333848576,"data_stored":333785361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390877184,"data_stored":390869633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:30.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:54:30.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'12322", 'reported_seq': 12406, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:23.857275+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:54:23.857275+0000', 'last_peered': '2024-09-16T04:54:23.857275+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:54:23.857275+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30375936, 'num_objects': 7416, 'num_object_clones': 0, 'num_object_copies': 14832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7416, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12322, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4075}, {'shards': '0,2', 'objects': 887}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:30.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:54:30.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'12466", 'reported_seq': 12609, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:28.208807+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:54:28.208807+0000', 'last_peered': '2024-09-16T04:54:28.208807+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:54:28.208807+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30273560, 'num_objects': 7392, 'num_object_clones': 0, 'num_object_copies': 14784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7392, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9205, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 784}, {'shards': '2', 'objects': 4073}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:30.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:54:30.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'12654", 'reported_seq': 12792, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:23.836949+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:54:23.836949+0000', 'last_peered': '2024-09-16T04:54:23.836949+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:54:23.836949+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31371264, 'num_objects': 7659, 'num_object_clones': 0, 'num_object_copies': 15318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7659, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9647, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4203}, {'shards': '0,2', 'objects': 961}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:30.284 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:54:30.284 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'12395", 'reported_seq': 12548, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:23.861939+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:54:23.861939+0000', 'last_peered': '2024-09-16T04:54:23.861939+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:54:23.861939+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30576640, 'num_objects': 7465, 'num_object_clones': 0, 'num_object_copies': 14930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7465, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9050, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4066}, {'shards': '0,2', 'objects': 937}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:30.284 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:54:30.284 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'12565", 'reported_seq': 12798, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:28.141818+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:54:28.141818+0000', 'last_peered': '2024-09-16T04:54:28.141818+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:54:28.141818+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30785536, 'num_objects': 7516, 'num_object_clones': 0, 'num_object_copies': 15032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7516, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9316, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 813}, {'shards': '2', 'objects': 4195}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:30.284 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:54:30.284 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'12617", 'reported_seq': 12752, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:23.810012+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:54:23.810012+0000', 'last_peered': '2024-09-16T04:54:23.810012+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:54:23.810012+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31133696, 'num_objects': 7601, 'num_object_clones': 0, 'num_object_copies': 15202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7601, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9272, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4136}, {'shards': '0,2', 'objects': 961}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:30.284 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:54:30.284 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'12493", 'reported_seq': 13646, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:28.188493+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:54:28.188493+0000', 'last_peered': '2024-09-16T04:54:28.188493+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:54:28.188493+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30663890, 'num_objects': 7488, 'num_object_clones': 0, 'num_object_copies': 14976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2038, 'num_objects_misplaced': 807, 'num_objects_unfound': 0, 'num_objects_dirty': 7488, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12465, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 455, 'num_bytes_recovered': 1847296, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 807}, {'shards': '2', 'objects': 4643}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:30.284 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:54:30.285 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'12500", 'reported_seq': 12603, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:28.099859+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:54:28.099859+0000', 'last_peered': '2024-09-16T04:54:28.099859+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:54:28.099859+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30212713, 'num_objects': 7377, 'num_object_clones': 0, 'num_object_copies': 14754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12499, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 846}, {'shards': '2', 'objects': 3970}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:30.285 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:54:30.285 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'12943", 'reported_seq': 13051, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:28.229358+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:54:28.229358+0000', 'last_peered': '2024-09-16T04:54:28.229358+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:54:28.229358+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31086395, 'num_objects': 7592, 'num_object_clones': 0, 'num_object_copies': 15184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7592, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12940, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 841}, {'shards': '2', 'objects': 4078}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:30.285 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:54:30.285 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'12804", 'reported_seq': 12949, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:23.900424+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:54:23.900424+0000', 'last_peered': '2024-09-16T04:54:23.900424+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:54:23.900424+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31179369, 'num_objects': 7613, 'num_object_clones': 0, 'num_object_copies': 15226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7613, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9573, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4089}, {'shards': '0,2', 'objects': 934}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:30.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:54:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:31.392 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32800/239158 objects degraded (13.715%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":419,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":119579,"data_bytes":489758986,"bytes_used":2117861376,"bytes_avail":287792431104,"bytes_total":289910292480,"degraded_objects":32800,"degraded_total":239158,"degraded_ratio":0.13714782695958319,"misplaced_objects":807,"misplaced_total":239158,"misplaced_ratio":0.0033743383035482817,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":63077,"recovering_keys_per_sec":0,"num_objects_recovered":185,"num_bytes_recovered":757760,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":463},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:54:31.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:54:31.425 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:32.326 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:32.328 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:32.465 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-16T04:54:30.406377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":489758986,"num_objects":119579,"num_object_clones":0,"num_object_copies":239158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32800,"num_objects_misplaced":807,"num_objects_unfound":0,"num_objects_dirty":119579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167055,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7136,"num_bytes_recovered":29175827,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160878,"ondisk_log_size":160878,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2068224,"kb_used_data":1088840,"kb_used_omap":12,"kb_used_meta":979315,"kb_avail":281047296,"statfs":{"total":289910292480,"available":287792431104,"internally_reserved":0,"allocated":1114972160,"data_stored":1114070590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1002818561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,6,9,12,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22827008,"num_objects":-5573,"num_object_clones":0,"num_object_copies":-11146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":103,"num_objects_misplaced":-288,"num_objects_unfound":0,"num_objects_dirty":-5573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5573,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":757760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013185"},"pg_stats":[{"pgid":"2.f","version":"49'12457","reported_seq":12539,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.932667+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:28.932667+0000","last_peered":"2024-09-16T04:54:28.932667+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:28.932667+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29822976,"num_objects":7281,"num_object_clones":0,"num_object_copies":14562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12457,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3940},{"shards":"0,2","objects":752}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12551","reported_seq":12672,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:28.157955+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:28.157955+0000","last_peered":"2024-09-16T04:54:28.157955+0000","last_clean":"2024-09-16T04:54:28.157955+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:28.157955+0000","last_undegraded":"2024-09-16T04:54:28.157955+0000","last_fullsized":"2024-09-16T04:54:28.157955+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30486528,"num_objects":7443,"num_object_clones":0,"num_object_copies":14886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9520,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12466","reported_seq":12609,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.208807+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:28.208807+0000","last_peered":"2024-09-16T04:54:28.208807+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:28.208807+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30273560,"num_objects":7392,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9205,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":784},{"shards":"2","objects":4073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12731","reported_seq":12806,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:28.200413+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:28.200413+0000","last_peered":"2024-09-16T04:54:28.200413+0000","last_clean":"2024-09-16T04:54:28.200413+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:28.200413+0000","last_undegraded":"2024-09-16T04:54:28.200413+0000","last_fullsized":"2024-09-16T04:54:28.200413+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31182848,"num_objects":7613,"num_object_clones":0,"num_object_copies":15226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12731,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12621","reported_seq":12844,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:28.968922+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:28.968922+0000","last_peered":"2024-09-16T04:54:28.968922+0000","last_clean":"2024-09-16T04:54:28.968922+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:28.968922+0000","last_undegraded":"2024-09-16T04:54:28.968922+0000","last_fullsized":"2024-09-16T04:54:28.968922+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30814208,"num_objects":7523,"num_object_clones":0,"num_object_copies":15046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9653,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'12798","reported_seq":12936,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.900856+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:28.900856+0000","last_peered":"2024-09-16T04:54:28.900856+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:28.900856+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30781440,"num_objects":7515,"num_object_clones":0,"num_object_copies":15030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9791,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4059},{"shards":"0,2","objects":817}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12547","reported_seq":12700,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.911872+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:28.911872+0000","last_peered":"2024-09-16T04:54:28.911872+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:28.911872+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29954048,"num_objects":7313,"num_object_clones":0,"num_object_copies":14626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9202,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3914},{"shards":"0,2","objects":785}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12532","reported_seq":12753,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:28.924044+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:28.924044+0000","last_peered":"2024-09-16T04:54:28.924044+0000","last_clean":"2024-09-16T04:54:28.924044+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:28.924044+0000","last_undegraded":"2024-09-16T04:54:28.924044+0000","last_fullsized":"2024-09-16T04:54:28.924044+0000","mapping_epoch":34,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30654464,"num_objects":7484,"num_object_clones":0,"num_object_copies":14968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9602,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12565","reported_seq":12798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.141818+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:28.141818+0000","last_peered":"2024-09-16T04:54:28.141818+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:28.141818+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30785536,"num_objects":7516,"num_object_clones":0,"num_object_copies":15032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9316,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":813},{"shards":"2","objects":4195}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'12770","reported_seq":12905,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.900154+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:28.900154+0000","last_peered":"2024-09-16T04:54:28.900154+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:28.900154+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30507008,"num_objects":7448,"num_object_clones":0,"num_object_copies":14896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9425,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3983},{"shards":"0,2","objects":808}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556564+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556564+0000","last_peered":"2024-09-16T04:54:05.556564+0000","last_clean":"2024-09-16T04:54:05.556564+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556564+0000","last_undegraded":"2024-09-16T04:54:05.556564+0000","last_fullsized":"2024-09-16T04:54:05.556564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12493","reported_seq":13646,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.188493+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:28.188493+0000","last_peered":"2024-09-16T04:54:28.188493+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:28.188493+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30663890,"num_objects":7488,"num_object_clones":0,"num_object_copies":14976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2038,"num_objects_misplaced":807,"num_objects_unfound":0,"num_objects_dirty":7488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12465,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":455,"num_bytes_recovered":1847296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":807},{"shards":"2","objects":4643}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12512","reported_seq":19373,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:28.080837+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:28.080837+0000","last_peered":"2024-09-16T04:54:28.080837+0000","last_clean":"2024-09-16T04:54:28.080837+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:28.080837+0000","last_undegraded":"2024-09-16T04:54:28.080837+0000","last_fullsized":"2024-09-16T04:54:28.080837+0000","mapping_epoch":46,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30580736,"num_objects":7466,"num_object_clones":0,"num_object_copies":14932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9235,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12500","reported_seq":12603,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.099859+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:28.099859+0000","last_peered":"2024-09-16T04:54:28.099859+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:28.099859+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30212713,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12499,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":846},{"shards":"2","objects":3970}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'12943","reported_seq":13051,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.229358+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:28.229358+0000","last_peered":"2024-09-16T04:54:28.229358+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:28.229358+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31086395,"num_objects":7592,"num_object_clones":0,"num_object_copies":15184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12940,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":841},{"shards":"2","objects":4078}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12544","reported_seq":19343,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:27.330190+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:27.330190+0000","last_peered":"2024-09-16T04:54:27.330190+0000","last_clean":"2024-09-16T04:54:27.330190+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:27.330190+0000","last_undegraded":"2024-09-16T04:54:27.330190+0000","last_fullsized":"2024-09-16T04:54:27.330190+0000","mapping_epoch":48,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31350784,"num_objects":7654,"num_object_clones":0,"num_object_copies":15308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9298,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'12945","reported_seq":13090,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:28.900855+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:28.900855+0000","last_peered":"2024-09-16T04:54:28.900855+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:28.900855+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30601833,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9714,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3948},{"shards":"0,2","objects":793}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":489758967,"num_objects":119577,"num_object_clones":0,"num_object_copies":239154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32800,"num_objects_misplaced":807,"num_objects_unfound":0,"num_objects_dirty":119577,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":167053,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7135,"num_bytes_recovered":29175808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1113137152,"data_stored":1113062907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716688,"kb_used_data":382332,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93655152,"statfs":{"total":96636764160,"available":95902875648,"internally_reserved":0,"allocated":391507968,"data_stored":391224642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,3,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593787,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637336,"kb_used_data":326596,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93734504,"statfs":{"total":96636764160,"available":95984132096,"internally_reserved":0,"allocated":334434304,"data_stored":334095314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":714200,"kb_used_data":379912,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93657640,"statfs":{"total":96636764160,"available":95905423360,"internally_reserved":0,"allocated":389029888,"data_stored":388750634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,4,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":388411392,"data_stored":388407913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":333848576,"data_stored":333785361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390877184,"data_stored":390869633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:35.467 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:36.377 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:36.380 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:36.442 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-16T04:54:34.407374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":480207114,"num_objects":117247,"num_object_clones":0,"num_object_copies":234494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32875,"num_objects_misplaced":650,"num_objects_unfound":0,"num_objects_dirty":117247,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169387,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7218,"num_bytes_recovered":29511699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2055716,"kb_used_data":1076324,"kb_used_omap":12,"kb_used_meta":979315,"kb_avail":281059804,"statfs":{"total":289910292480,"available":287805239296,"internally_reserved":0,"allocated":1102155776,"data_stored":1101254206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1002818561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,9,4,4,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27836416,"num_objects":-6796,"num_object_clones":0,"num_object_copies":-13592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":178,"num_objects_misplaced":-445,"num_objects_unfound":0,"num_objects_dirty":-6796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6796,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":1093632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011929"},"pg_stats":[{"pgid":"2.f","version":"49'12592","reported_seq":12676,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.957626+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:33.957626+0000","last_peered":"2024-09-16T04:54:33.957626+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:33.957626+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29270016,"num_objects":7146,"num_object_clones":0,"num_object_copies":14292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12592,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3805},{"shards":"0,2","objects":617}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12711","reported_seq":12832,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.300076+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:33.300076+0000","last_peered":"2024-09-16T04:54:33.300076+0000","last_clean":"2024-09-16T04:54:33.300076+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:33.300076+0000","last_undegraded":"2024-09-16T04:54:33.300076+0000","last_fullsized":"2024-09-16T04:54:33.300076+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29831168,"num_objects":7283,"num_object_clones":0,"num_object_copies":14566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9680,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12606","reported_seq":12750,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.172668+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:33.172668+0000","last_peered":"2024-09-16T04:54:33.172668+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:33.172668+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29700120,"num_objects":7252,"num_object_clones":0,"num_object_copies":14504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7252,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9345,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":644},{"shards":"2","objects":3933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12853","reported_seq":12928,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.279023+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:33.279023+0000","last_peered":"2024-09-16T04:54:33.279023+0000","last_clean":"2024-09-16T04:54:33.279023+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:33.279023+0000","last_undegraded":"2024-09-16T04:54:33.279023+0000","last_fullsized":"2024-09-16T04:54:33.279023+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30683136,"num_objects":7491,"num_object_clones":0,"num_object_copies":14982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12853,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12774","reported_seq":12998,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.928990+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:33.928990+0000","last_peered":"2024-09-16T04:54:33.928990+0000","last_clean":"2024-09-16T04:54:33.928990+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:33.928990+0000","last_undegraded":"2024-09-16T04:54:33.928990+0000","last_fullsized":"2024-09-16T04:54:33.928990+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30187520,"num_objects":7370,"num_object_clones":0,"num_object_copies":14740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9806,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'12936","reported_seq":13074,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.877990+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:33.877990+0000","last_peered":"2024-09-16T04:54:33.877990+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:33.877990+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30216192,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9929,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":679}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12701","reported_seq":12854,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.823975+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:33.823975+0000","last_peered":"2024-09-16T04:54:33.823975+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:33.823975+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29323264,"num_objects":7159,"num_object_clones":0,"num_object_copies":14318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9356,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3760},{"shards":"0,2","objects":631}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12673","reported_seq":12894,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.953186+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:33.953186+0000","last_peered":"2024-09-16T04:54:33.953186+0000","last_clean":"2024-09-16T04:54:33.953186+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:33.953186+0000","last_undegraded":"2024-09-16T04:54:33.953186+0000","last_fullsized":"2024-09-16T04:54:33.953186+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30076928,"num_objects":7343,"num_object_clones":0,"num_object_copies":14686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9743,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12701","reported_seq":12933,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.311318+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:33.311318+0000","last_peered":"2024-09-16T04:54:33.311318+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:33.311318+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30228480,"num_objects":7380,"num_object_clones":0,"num_object_copies":14760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9452,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":677},{"shards":"2","objects":4059}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'12922","reported_seq":13057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.892418+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:33.892418+0000","last_peered":"2024-09-16T04:54:33.892418+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:33.892418+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9577,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3831},{"shards":"0,2","objects":656}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556564+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556564+0000","last_peered":"2024-09-16T04:54:05.556564+0000","last_clean":"2024-09-16T04:54:05.556564+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556564+0000","last_undegraded":"2024-09-16T04:54:05.556564+0000","last_fullsized":"2024-09-16T04:54:05.556564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12646","reported_seq":13964,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.302549+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:33.302549+0000","last_peered":"2024-09-16T04:54:33.302549+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:33.302549+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30037202,"num_objects":7335,"num_object_clones":0,"num_object_copies":14670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2113,"num_objects_misplaced":650,"num_objects_unfound":0,"num_objects_dirty":7335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12618,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2183168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":650},{"shards":"2","objects":4572}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12678","reported_seq":19539,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.246932+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:33.246932+0000","last_peered":"2024-09-16T04:54:33.246932+0000","last_clean":"2024-09-16T04:54:33.246932+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:33.246932+0000","last_undegraded":"2024-09-16T04:54:33.246932+0000","last_fullsized":"2024-09-16T04:54:33.246932+0000","mapping_epoch":46,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29900800,"num_objects":7300,"num_object_clones":0,"num_object_copies":14600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9401,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12635","reported_seq":12738,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.308551+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:33.308551+0000","last_peered":"2024-09-16T04:54:33.308551+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:33.308551+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29659753,"num_objects":7242,"num_object_clones":0,"num_object_copies":14484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12634,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":711},{"shards":"2","objects":3835}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13086","reported_seq":13193,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.266559+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:33.266559+0000","last_peered":"2024-09-16T04:54:33.266559+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:33.266559+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30500667,"num_objects":7449,"num_object_clones":0,"num_object_copies":14898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13083,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":698},{"shards":"2","objects":3935}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12705","reported_seq":19503,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:32.477907+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:32.477907+0000","last_peered":"2024-09-16T04:54:32.477907+0000","last_clean":"2024-09-16T04:54:32.477907+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:32.477907+0000","last_undegraded":"2024-09-16T04:54:32.477907+0000","last_fullsized":"2024-09-16T04:54:32.477907+0000","mapping_epoch":48,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30691328,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9459,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13088","reported_seq":13233,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.824389+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:33.824389+0000","last_peered":"2024-09-16T04:54:33.824389+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:33.824389+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30016105,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9857,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3805},{"shards":"0,2","objects":650}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":480207095,"num_objects":117245,"num_object_clones":0,"num_object_copies":234490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32875,"num_objects_misplaced":650,"num_objects_unfound":0,"num_objects_dirty":117245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169385,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7217,"num_bytes_recovered":29511680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1099952128,"data_stored":1099877883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160707,"ondisk_log_size":160707,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":715328,"kb_used_data":376812,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93656512,"statfs":{"total":96636764160,"available":95904268288,"internally_reserved":0,"allocated":385855488,"data_stored":385572162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593788,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627700,"kb_used_data":325284,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93744140,"statfs":{"total":96636764160,"available":95993999360,"internally_reserved":0,"allocated":333090816,"data_stored":332751826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,1,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712688,"kb_used_data":374228,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93659152,"statfs":{"total":96636764160,"available":95906971648,"internally_reserved":0,"allocated":383209472,"data_stored":382930218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382517248,"data_stored":382513769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":332472320,"data_stored":332409105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":384962560,"data_stored":384955009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:36.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:37.254 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:37.295 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-16T04:54:36.407895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":480207114,"num_objects":117247,"num_object_clones":0,"num_object_copies":234494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32875,"num_objects_misplaced":650,"num_objects_unfound":0,"num_objects_dirty":117247,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169387,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7218,"num_bytes_recovered":29511699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2055716,"kb_used_data":1076324,"kb_used_omap":12,"kb_used_meta":979315,"kb_avail":281059804,"statfs":{"total":289910292480,"available":287805239296,"internally_reserved":0,"allocated":1102155776,"data_stored":1101254206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1002818561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,9,4,4,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18800640,"num_objects":-4590,"num_object_clones":0,"num_object_copies":-9180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":135,"num_objects_misplaced":-303,"num_objects_unfound":0,"num_objects_dirty":-4590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4590,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003105"},"pg_stats":[{"pgid":"2.f","version":"49'12592","reported_seq":12676,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.957626+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:33.957626+0000","last_peered":"2024-09-16T04:54:33.957626+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:33.957626+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29270016,"num_objects":7146,"num_object_clones":0,"num_object_copies":14292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12592,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3805},{"shards":"0,2","objects":617}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12711","reported_seq":12832,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.300076+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:33.300076+0000","last_peered":"2024-09-16T04:54:33.300076+0000","last_clean":"2024-09-16T04:54:33.300076+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:33.300076+0000","last_undegraded":"2024-09-16T04:54:33.300076+0000","last_fullsized":"2024-09-16T04:54:33.300076+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29831168,"num_objects":7283,"num_object_clones":0,"num_object_copies":14566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9680,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12606","reported_seq":12750,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.172668+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:33.172668+0000","last_peered":"2024-09-16T04:54:33.172668+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:33.172668+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29700120,"num_objects":7252,"num_object_clones":0,"num_object_copies":14504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7252,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9345,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":644},{"shards":"2","objects":3933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12853","reported_seq":12928,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.279023+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:33.279023+0000","last_peered":"2024-09-16T04:54:33.279023+0000","last_clean":"2024-09-16T04:54:33.279023+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:33.279023+0000","last_undegraded":"2024-09-16T04:54:33.279023+0000","last_fullsized":"2024-09-16T04:54:33.279023+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30683136,"num_objects":7491,"num_object_clones":0,"num_object_copies":14982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12853,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12774","reported_seq":12998,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.928990+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:33.928990+0000","last_peered":"2024-09-16T04:54:33.928990+0000","last_clean":"2024-09-16T04:54:33.928990+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:33.928990+0000","last_undegraded":"2024-09-16T04:54:33.928990+0000","last_fullsized":"2024-09-16T04:54:33.928990+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30187520,"num_objects":7370,"num_object_clones":0,"num_object_copies":14740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9806,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'12936","reported_seq":13074,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.877990+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:33.877990+0000","last_peered":"2024-09-16T04:54:33.877990+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:33.877990+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30216192,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9929,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":679}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12701","reported_seq":12854,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.823975+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:33.823975+0000","last_peered":"2024-09-16T04:54:33.823975+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:33.823975+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29323264,"num_objects":7159,"num_object_clones":0,"num_object_copies":14318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9356,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3760},{"shards":"0,2","objects":631}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12673","reported_seq":12894,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.953186+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:33.953186+0000","last_peered":"2024-09-16T04:54:33.953186+0000","last_clean":"2024-09-16T04:54:33.953186+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:33.953186+0000","last_undegraded":"2024-09-16T04:54:33.953186+0000","last_fullsized":"2024-09-16T04:54:33.953186+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30076928,"num_objects":7343,"num_object_clones":0,"num_object_copies":14686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9743,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12701","reported_seq":12933,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.311318+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:33.311318+0000","last_peered":"2024-09-16T04:54:33.311318+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:33.311318+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30228480,"num_objects":7380,"num_object_clones":0,"num_object_copies":14760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9452,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":677},{"shards":"2","objects":4059}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'12922","reported_seq":13057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.892418+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:33.892418+0000","last_peered":"2024-09-16T04:54:33.892418+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:33.892418+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9577,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3831},{"shards":"0,2","objects":656}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556564+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556564+0000","last_peered":"2024-09-16T04:54:05.556564+0000","last_clean":"2024-09-16T04:54:05.556564+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556564+0000","last_undegraded":"2024-09-16T04:54:05.556564+0000","last_fullsized":"2024-09-16T04:54:05.556564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12646","reported_seq":13964,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.302549+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:33.302549+0000","last_peered":"2024-09-16T04:54:33.302549+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:33.302549+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30037202,"num_objects":7335,"num_object_clones":0,"num_object_copies":14670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2113,"num_objects_misplaced":650,"num_objects_unfound":0,"num_objects_dirty":7335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12618,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2183168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":650},{"shards":"2","objects":4572}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12678","reported_seq":19539,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.246932+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:33.246932+0000","last_peered":"2024-09-16T04:54:33.246932+0000","last_clean":"2024-09-16T04:54:33.246932+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:33.246932+0000","last_undegraded":"2024-09-16T04:54:33.246932+0000","last_fullsized":"2024-09-16T04:54:33.246932+0000","mapping_epoch":46,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29900800,"num_objects":7300,"num_object_clones":0,"num_object_copies":14600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9401,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12635","reported_seq":12738,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.308551+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:33.308551+0000","last_peered":"2024-09-16T04:54:33.308551+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:33.308551+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29659753,"num_objects":7242,"num_object_clones":0,"num_object_copies":14484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12634,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":711},{"shards":"2","objects":3835}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13086","reported_seq":13193,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.266559+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:33.266559+0000","last_peered":"2024-09-16T04:54:33.266559+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:33.266559+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30500667,"num_objects":7449,"num_object_clones":0,"num_object_copies":14898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13083,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":698},{"shards":"2","objects":3935}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12705","reported_seq":19503,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:32.477907+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:32.477907+0000","last_peered":"2024-09-16T04:54:32.477907+0000","last_clean":"2024-09-16T04:54:32.477907+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:32.477907+0000","last_undegraded":"2024-09-16T04:54:32.477907+0000","last_fullsized":"2024-09-16T04:54:32.477907+0000","mapping_epoch":48,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30691328,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9459,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13088","reported_seq":13233,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.824389+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:33.824389+0000","last_peered":"2024-09-16T04:54:33.824389+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:33.824389+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30016105,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9857,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3805},{"shards":"0,2","objects":650}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":480207095,"num_objects":117245,"num_object_clones":0,"num_object_copies":234490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32875,"num_objects_misplaced":650,"num_objects_unfound":0,"num_objects_dirty":117245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169385,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7217,"num_bytes_recovered":29511680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1099952128,"data_stored":1099877883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160707,"ondisk_log_size":160707,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":715328,"kb_used_data":376812,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93656512,"statfs":{"total":96636764160,"available":95904268288,"internally_reserved":0,"allocated":385855488,"data_stored":385572162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593788,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627700,"kb_used_data":325284,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93744140,"statfs":{"total":96636764160,"available":95993999360,"internally_reserved":0,"allocated":333090816,"data_stored":332751826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,1,1,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712688,"kb_used_data":374228,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93659152,"statfs":{"total":96636764160,"available":95906971648,"internally_reserved":0,"allocated":383209472,"data_stored":382930218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382517248,"data_stored":382513769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":332472320,"data_stored":332409105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":384962560,"data_stored":384955009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:37.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:54:37.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'12592", 'reported_seq': 12676, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:33.957626+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:54:33.957626+0000', 'last_peered': '2024-09-16T04:54:33.957626+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:54:33.957626+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29270016, 'num_objects': 7146, 'num_object_clones': 0, 'num_object_copies': 14292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7146, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12592, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3805}, {'shards': '0,2', 'objects': 617}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:37.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:54:37.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'12606", 'reported_seq': 12750, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:33.172668+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:54:33.172668+0000', 'last_peered': '2024-09-16T04:54:33.172668+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:54:33.172668+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29700120, 'num_objects': 7252, 'num_object_clones': 0, 'num_object_copies': 14504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7252, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9345, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 644}, {'shards': '2', 'objects': 3933}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:37.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:54:37.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'12936", 'reported_seq': 13074, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:33.877990+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:54:33.877990+0000', 'last_peered': '2024-09-16T04:54:33.877990+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:54:33.877990+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30216192, 'num_objects': 7377, 'num_object_clones': 0, 'num_object_copies': 14754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9929, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3921}, {'shards': '0,2', 'objects': 679}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:37.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:54:37.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'12701", 'reported_seq': 12854, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:33.823975+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:54:33.823975+0000', 'last_peered': '2024-09-16T04:54:33.823975+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:54:33.823975+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29323264, 'num_objects': 7159, 'num_object_clones': 0, 'num_object_copies': 14318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9356, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3760}, {'shards': '0,2', 'objects': 631}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:37.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:54:37.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'12701", 'reported_seq': 12933, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:33.311318+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:54:33.311318+0000', 'last_peered': '2024-09-16T04:54:33.311318+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:54:33.311318+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30228480, 'num_objects': 7380, 'num_object_clones': 0, 'num_object_copies': 14760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7380, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9452, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 677}, {'shards': '2', 'objects': 4059}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:37.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:54:37.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'12922", 'reported_seq': 13057, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:33.892418+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:54:33.892418+0000', 'last_peered': '2024-09-16T04:54:33.892418+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:54:33.892418+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29884416, 'num_objects': 7296, 'num_object_clones': 0, 'num_object_copies': 14592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9577, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3831}, {'shards': '0,2', 'objects': 656}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:37.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:54:37.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'12646", 'reported_seq': 13964, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:33.302549+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:54:33.302549+0000', 'last_peered': '2024-09-16T04:54:33.302549+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:54:33.302549+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30037202, 'num_objects': 7335, 'num_object_clones': 0, 'num_object_copies': 14670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2113, 'num_objects_misplaced': 650, 'num_objects_unfound': 0, 'num_objects_dirty': 7335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12618, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 537, 'num_bytes_recovered': 2183168, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 650}, {'shards': '2', 'objects': 4572}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:37.297 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:54:37.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'12635", 'reported_seq': 12738, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:33.308551+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:54:33.308551+0000', 'last_peered': '2024-09-16T04:54:33.308551+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:54:33.308551+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29659753, 'num_objects': 7242, 'num_object_clones': 0, 'num_object_copies': 14484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7242, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12634, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 711}, {'shards': '2', 'objects': 3835}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:37.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:54:37.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'13086", 'reported_seq': 13193, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:33.266559+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:54:33.266559+0000', 'last_peered': '2024-09-16T04:54:33.266559+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:54:33.266559+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30500667, 'num_objects': 7449, 'num_object_clones': 0, 'num_object_copies': 14898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7449, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13083, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 698}, {'shards': '2', 'objects': 3935}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:37.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:54:37.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'13088", 'reported_seq': 13233, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:33.824389+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:54:33.824389+0000', 'last_peered': '2024-09-16T04:54:33.824389+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:54:33.824389+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30016105, 'num_objects': 7329, 'num_object_clones': 0, 'num_object_copies': 14658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7329, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9857, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3805}, {'shards': '0,2', 'objects': 650}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:37.298 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:54:38.258 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:38.259 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32875/234494 objects degraded (14.020%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":426,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":117247,"data_bytes":480207114,"bytes_used":2105053184,"bytes_avail":287805239296,"bytes_total":289910292480,"degraded_objects":32875,"degraded_total":234494,"degraded_ratio":0.14019548474587837,"misplaced_objects":650,"misplaced_total":234494,"misplaced_ratio":0.0027719259341390397,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":57329,"recovering_keys_per_sec":0,"num_objects_recovered":168,"num_bytes_recovered":688128,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":382},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:54:38.334 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:54:38.334 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:39.132 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:39.133 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:39.214 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-16T04:54:38.408338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":479600906,"num_objects":117099,"num_object_clones":0,"num_object_copies":234198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32875,"num_objects_misplaced":650,"num_objects_unfound":0,"num_objects_dirty":117099,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169535,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7218,"num_bytes_recovered":29511699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2058728,"kb_used_data":1075176,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281056792,"statfs":{"total":289910292480,"available":287802155008,"internally_reserved":0,"allocated":1100980224,"data_stored":1100078654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1007078401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,11,4,5,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19406848,"num_objects":-4738,"num_object_clones":0,"num_object_copies":-9476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":135,"num_objects_misplaced":-303,"num_objects_unfound":0,"num_objects_dirty":-4738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4738,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002984"},"pg_stats":[{"pgid":"2.f","version":"49'12592","reported_seq":12676,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.957626+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:33.957626+0000","last_peered":"2024-09-16T04:54:33.957626+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:33.957626+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29270016,"num_objects":7146,"num_object_clones":0,"num_object_copies":14292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12592,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3805},{"shards":"0,2","objects":617}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12711","reported_seq":12832,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.300076+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:33.300076+0000","last_peered":"2024-09-16T04:54:33.300076+0000","last_clean":"2024-09-16T04:54:33.300076+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:33.300076+0000","last_undegraded":"2024-09-16T04:54:33.300076+0000","last_fullsized":"2024-09-16T04:54:33.300076+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29831168,"num_objects":7283,"num_object_clones":0,"num_object_copies":14566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9680,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12606","reported_seq":12750,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.172668+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:33.172668+0000","last_peered":"2024-09-16T04:54:33.172668+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:33.172668+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29700120,"num_objects":7252,"num_object_clones":0,"num_object_copies":14504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7252,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9345,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":644},{"shards":"2","objects":3933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'12853","reported_seq":12928,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.279023+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:33.279023+0000","last_peered":"2024-09-16T04:54:33.279023+0000","last_clean":"2024-09-16T04:54:33.279023+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:33.279023+0000","last_undegraded":"2024-09-16T04:54:33.279023+0000","last_fullsized":"2024-09-16T04:54:33.279023+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30683136,"num_objects":7491,"num_object_clones":0,"num_object_copies":14982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12853,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12774","reported_seq":12998,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.928990+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:33.928990+0000","last_peered":"2024-09-16T04:54:33.928990+0000","last_clean":"2024-09-16T04:54:33.928990+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:33.928990+0000","last_undegraded":"2024-09-16T04:54:33.928990+0000","last_fullsized":"2024-09-16T04:54:33.928990+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30187520,"num_objects":7370,"num_object_clones":0,"num_object_copies":14740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9806,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'12936","reported_seq":13074,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.877990+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:33.877990+0000","last_peered":"2024-09-16T04:54:33.877990+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:33.877990+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30216192,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9929,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":679}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12701","reported_seq":12854,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.823975+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:33.823975+0000","last_peered":"2024-09-16T04:54:33.823975+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:33.823975+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29323264,"num_objects":7159,"num_object_clones":0,"num_object_copies":14318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9356,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3760},{"shards":"0,2","objects":631}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12673","reported_seq":12894,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.953186+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:33.953186+0000","last_peered":"2024-09-16T04:54:33.953186+0000","last_clean":"2024-09-16T04:54:33.953186+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:33.953186+0000","last_undegraded":"2024-09-16T04:54:33.953186+0000","last_fullsized":"2024-09-16T04:54:33.953186+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30076928,"num_objects":7343,"num_object_clones":0,"num_object_copies":14686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9743,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12701","reported_seq":12933,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.311318+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:33.311318+0000","last_peered":"2024-09-16T04:54:33.311318+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:33.311318+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30228480,"num_objects":7380,"num_object_clones":0,"num_object_copies":14760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9452,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":677},{"shards":"2","objects":4059}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'12922","reported_seq":13057,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.892418+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:33.892418+0000","last_peered":"2024-09-16T04:54:33.892418+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:33.892418+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29884416,"num_objects":7296,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9577,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3831},{"shards":"0,2","objects":656}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556564+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556564+0000","last_peered":"2024-09-16T04:54:05.556564+0000","last_clean":"2024-09-16T04:54:05.556564+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556564+0000","last_undegraded":"2024-09-16T04:54:05.556564+0000","last_fullsized":"2024-09-16T04:54:05.556564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12646","reported_seq":13964,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.302549+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:33.302549+0000","last_peered":"2024-09-16T04:54:33.302549+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:33.302549+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30037202,"num_objects":7335,"num_object_clones":0,"num_object_copies":14670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2113,"num_objects_misplaced":650,"num_objects_unfound":0,"num_objects_dirty":7335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12618,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2183168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":650},{"shards":"2","objects":4572}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12678","reported_seq":19539,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:33.246932+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:33.246932+0000","last_peered":"2024-09-16T04:54:33.246932+0000","last_clean":"2024-09-16T04:54:33.246932+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:33.246932+0000","last_undegraded":"2024-09-16T04:54:33.246932+0000","last_fullsized":"2024-09-16T04:54:33.246932+0000","mapping_epoch":46,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29900800,"num_objects":7300,"num_object_clones":0,"num_object_copies":14600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9401,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12635","reported_seq":12738,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.308551+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:33.308551+0000","last_peered":"2024-09-16T04:54:33.308551+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:33.308551+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29659753,"num_objects":7242,"num_object_clones":0,"num_object_copies":14484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12634,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":711},{"shards":"2","objects":3835}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13086","reported_seq":13193,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.266559+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:33.266559+0000","last_peered":"2024-09-16T04:54:33.266559+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:33.266559+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30500667,"num_objects":7449,"num_object_clones":0,"num_object_copies":14898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13083,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":698},{"shards":"2","objects":3935}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12853","reported_seq":19652,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:37.520509+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:37.520509+0000","last_peered":"2024-09-16T04:54:37.520509+0000","last_clean":"2024-09-16T04:54:37.520509+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:37.520509+0000","last_undegraded":"2024-09-16T04:54:37.520509+0000","last_fullsized":"2024-09-16T04:54:37.520509+0000","mapping_epoch":48,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30085120,"num_objects":7345,"num_object_clones":0,"num_object_copies":14690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9607,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13088","reported_seq":13233,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:33.824389+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:33.824389+0000","last_peered":"2024-09-16T04:54:33.824389+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:33.824389+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30016105,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9857,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3805},{"shards":"0,2","objects":650}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":479600887,"num_objects":117097,"num_object_clones":0,"num_object_copies":234194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32875,"num_objects_misplaced":650,"num_objects_unfound":0,"num_objects_dirty":117097,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":169533,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7217,"num_bytes_recovered":29511680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1098862592,"data_stored":1098788347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":715328,"kb_used_data":376812,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93656512,"statfs":{"total":96636764160,"available":95904268288,"internally_reserved":0,"allocated":385855488,"data_stored":385572162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593789,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630712,"kb_used_data":324136,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93741128,"statfs":{"total":96636764160,"available":95990915072,"internally_reserved":0,"allocated":331915264,"data_stored":331576274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,2,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712688,"kb_used_data":374228,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93659152,"statfs":{"total":96636764160,"available":95906971648,"internally_reserved":0,"allocated":383209472,"data_stored":382930218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":382517248,"data_stored":382513769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331382784,"data_stored":331319569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":384962560,"data_stored":384955009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:42.217 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:43.027 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:43.028 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:43.126 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-16T04:54:42.409337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":470774026,"num_objects":114944,"num_object_clones":0,"num_object_copies":229888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32931,"num_objects_misplaced":506,"num_objects_unfound":0,"num_objects_dirty":114944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171690,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7306,"num_bytes_recovered":29872147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2059540,"kb_used_data":1063512,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":281055980,"statfs":{"total":289910292480,"available":287801323520,"internally_reserved":0,"allocated":1089036288,"data_stored":1088134718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1019857921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,6,9,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18984960,"num_objects":-4635,"num_object_clones":0,"num_object_copies":-9270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":-301,"num_objects_unfound":0,"num_objects_dirty":-4635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4635,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":696320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002960"},"pg_stats":[{"pgid":"2.f","version":"49'12729","reported_seq":12813,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.766470+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:38.766470+0000","last_peered":"2024-09-16T04:54:38.766470+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:38.766470+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28708864,"num_objects":7009,"num_object_clones":0,"num_object_copies":14018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12729,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3668},{"shards":"0,2","objects":480}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12859","reported_seq":12980,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:38.341937+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:38.341937+0000","last_peered":"2024-09-16T04:54:38.341937+0000","last_clean":"2024-09-16T04:54:38.341937+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:38.341937+0000","last_undegraded":"2024-09-16T04:54:38.341937+0000","last_fullsized":"2024-09-16T04:54:38.341937+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29224960,"num_objects":7135,"num_object_clones":0,"num_object_copies":14270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9828,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12736","reported_seq":12880,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.249328+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:38.249328+0000","last_peered":"2024-09-16T04:54:38.249328+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:38.249328+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29167640,"num_objects":7122,"num_object_clones":0,"num_object_copies":14244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9475,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":514},{"shards":"2","objects":3803}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13006","reported_seq":13081,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:38.338710+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:38.338710+0000","last_peered":"2024-09-16T04:54:38.338710+0000","last_clean":"2024-09-16T04:54:38.338710+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:38.338710+0000","last_undegraded":"2024-09-16T04:54:38.338710+0000","last_fullsized":"2024-09-16T04:54:38.338710+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30056448,"num_objects":7338,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13006,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12928","reported_seq":13152,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:38.977260+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:38.977260+0000","last_peered":"2024-09-16T04:54:38.977260+0000","last_clean":"2024-09-16T04:54:38.977260+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:38.977260+0000","last_undegraded":"2024-09-16T04:54:38.977260+0000","last_fullsized":"2024-09-16T04:54:38.977260+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29556736,"num_objects":7216,"num_object_clones":0,"num_object_copies":14432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9960,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13064","reported_seq":13202,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.904893+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:38.904893+0000","last_peered":"2024-09-16T04:54:38.904893+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:38.904893+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29691904,"num_objects":7249,"num_object_clones":0,"num_object_copies":14498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10057,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3793},{"shards":"0,2","objects":551}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12845","reported_seq":12998,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.993454+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:38.993454+0000","last_peered":"2024-09-16T04:54:38.993454+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:38.993454+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28733440,"num_objects":7015,"num_object_clones":0,"num_object_copies":14030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9500,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3616},{"shards":"0,2","objects":487}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12809","reported_seq":13030,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:38.892854+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:38.892854+0000","last_peered":"2024-09-16T04:54:38.892854+0000","last_clean":"2024-09-16T04:54:38.892854+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:38.892854+0000","last_undegraded":"2024-09-16T04:54:38.892854+0000","last_fullsized":"2024-09-16T04:54:38.892854+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29519872,"num_objects":7207,"num_object_clones":0,"num_object_copies":14414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9879,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12864","reported_seq":13097,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.235793+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:38.235793+0000","last_peered":"2024-09-16T04:54:38.235793+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:38.235793+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29560832,"num_objects":7217,"num_object_clones":0,"num_object_copies":14434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9615,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":514},{"shards":"2","objects":3896}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13076","reported_seq":13211,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.963037+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:38.963037+0000","last_peered":"2024-09-16T04:54:38.963037+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:38.963037+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29253632,"num_objects":7142,"num_object_clones":0,"num_object_copies":14284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9731,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3677},{"shards":"0,2","objects":502}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12794","reported_seq":14288,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.365114+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:38.365114+0000","last_peered":"2024-09-16T04:54:38.365114+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:38.365114+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29430994,"num_objects":7187,"num_object_clones":0,"num_object_copies":14374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2169,"num_objects_misplaced":506,"num_objects_unfound":0,"num_objects_dirty":7187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12766,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":625,"num_bytes_recovered":2543616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":506},{"shards":"2","objects":4512}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12802","reported_seq":19660,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:38.230369+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:38.230369+0000","last_peered":"2024-09-16T04:54:38.230369+0000","last_clean":"2024-09-16T04:54:38.230369+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:38.230369+0000","last_undegraded":"2024-09-16T04:54:38.230369+0000","last_fullsized":"2024-09-16T04:54:38.230369+0000","mapping_epoch":46,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29392896,"num_objects":7176,"num_object_clones":0,"num_object_copies":14352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9525,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12789","reported_seq":12892,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.328028+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:38.328028+0000","last_peered":"2024-09-16T04:54:38.328028+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:38.328028+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29028969,"num_objects":7088,"num_object_clones":0,"num_object_copies":14176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12788,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":557},{"shards":"2","objects":3681}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13233","reported_seq":13341,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.324009+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:38.324009+0000","last_peered":"2024-09-16T04:54:38.324009+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:38.324009+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29898555,"num_objects":7302,"num_object_clones":0,"num_object_copies":14604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13230,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":551},{"shards":"2","objects":3788}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12853","reported_seq":19652,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:37.520509+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:37.520509+0000","last_peered":"2024-09-16T04:54:37.520509+0000","last_clean":"2024-09-16T04:54:37.520509+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:37.520509+0000","last_undegraded":"2024-09-16T04:54:37.520509+0000","last_fullsized":"2024-09-16T04:54:37.520509+0000","mapping_epoch":48,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30085120,"num_objects":7345,"num_object_clones":0,"num_object_copies":14690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9607,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13223","reported_seq":13368,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.936532+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:38.936532+0000","last_peered":"2024-09-16T04:54:38.936532+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:38.936532+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29463145,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9992,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3670},{"shards":"0,2","objects":515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":470774007,"num_objects":114942,"num_object_clones":0,"num_object_copies":229884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32931,"num_objects_misplaced":506,"num_objects_unfound":0,"num_objects_dirty":114942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171688,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7305,"num_bytes_recovered":29872128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1087004672,"data_stored":1086930427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717816,"kb_used_data":370980,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93654024,"statfs":{"total":96636764160,"available":95901720576,"internally_reserved":0,"allocated":379883520,"data_stored":379600194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593789,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630712,"kb_used_data":324136,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93741128,"statfs":{"total":96636764160,"available":95990915072,"internally_reserved":0,"allocated":331915264,"data_stored":331576274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,2,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711012,"kb_used_data":368396,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93660828,"statfs":{"total":96636764160,"available":95908687872,"internally_reserved":0,"allocated":377237504,"data_stored":376958250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":376729600,"data_stored":376726121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331382784,"data_stored":331319569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378892288,"data_stored":378884737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:43.128 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:43.989 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:44.054 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-16T04:54:42.409337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":470774026,"num_objects":114944,"num_object_clones":0,"num_object_copies":229888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32931,"num_objects_misplaced":506,"num_objects_unfound":0,"num_objects_dirty":114944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171690,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7306,"num_bytes_recovered":29872147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2059540,"kb_used_data":1063512,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":281055980,"statfs":{"total":289910292480,"available":287801323520,"internally_reserved":0,"allocated":1089036288,"data_stored":1088134718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1019857921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,6,9,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18984960,"num_objects":-4635,"num_object_clones":0,"num_object_copies":-9270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":131,"num_objects_misplaced":-301,"num_objects_unfound":0,"num_objects_dirty":-4635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4635,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":696320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002960"},"pg_stats":[{"pgid":"2.f","version":"49'12729","reported_seq":12813,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.766470+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:38.766470+0000","last_peered":"2024-09-16T04:54:38.766470+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:38.766470+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28708864,"num_objects":7009,"num_object_clones":0,"num_object_copies":14018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12729,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3668},{"shards":"0,2","objects":480}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'12859","reported_seq":12980,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:38.341937+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:38.341937+0000","last_peered":"2024-09-16T04:54:38.341937+0000","last_clean":"2024-09-16T04:54:38.341937+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:38.341937+0000","last_undegraded":"2024-09-16T04:54:38.341937+0000","last_fullsized":"2024-09-16T04:54:38.341937+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29224960,"num_objects":7135,"num_object_clones":0,"num_object_copies":14270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9828,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12736","reported_seq":12880,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.249328+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:38.249328+0000","last_peered":"2024-09-16T04:54:38.249328+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:38.249328+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29167640,"num_objects":7122,"num_object_clones":0,"num_object_copies":14244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9475,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":514},{"shards":"2","objects":3803}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13006","reported_seq":13081,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:38.338710+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:38.338710+0000","last_peered":"2024-09-16T04:54:38.338710+0000","last_clean":"2024-09-16T04:54:38.338710+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:38.338710+0000","last_undegraded":"2024-09-16T04:54:38.338710+0000","last_fullsized":"2024-09-16T04:54:38.338710+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30056448,"num_objects":7338,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13006,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'12928","reported_seq":13152,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:38.977260+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:38.977260+0000","last_peered":"2024-09-16T04:54:38.977260+0000","last_clean":"2024-09-16T04:54:38.977260+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:38.977260+0000","last_undegraded":"2024-09-16T04:54:38.977260+0000","last_fullsized":"2024-09-16T04:54:38.977260+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29556736,"num_objects":7216,"num_object_clones":0,"num_object_copies":14432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9960,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13064","reported_seq":13202,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.904893+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:38.904893+0000","last_peered":"2024-09-16T04:54:38.904893+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:38.904893+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29691904,"num_objects":7249,"num_object_clones":0,"num_object_copies":14498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10057,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3793},{"shards":"0,2","objects":551}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12845","reported_seq":12998,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.993454+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:38.993454+0000","last_peered":"2024-09-16T04:54:38.993454+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:38.993454+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28733440,"num_objects":7015,"num_object_clones":0,"num_object_copies":14030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9500,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3616},{"shards":"0,2","objects":487}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12809","reported_seq":13030,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:38.892854+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:38.892854+0000","last_peered":"2024-09-16T04:54:38.892854+0000","last_clean":"2024-09-16T04:54:38.892854+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:38.892854+0000","last_undegraded":"2024-09-16T04:54:38.892854+0000","last_fullsized":"2024-09-16T04:54:38.892854+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29519872,"num_objects":7207,"num_object_clones":0,"num_object_copies":14414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9879,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'12864","reported_seq":13097,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.235793+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:38.235793+0000","last_peered":"2024-09-16T04:54:38.235793+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:38.235793+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29560832,"num_objects":7217,"num_object_clones":0,"num_object_copies":14434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9615,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":514},{"shards":"2","objects":3896}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13076","reported_seq":13211,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.963037+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:38.963037+0000","last_peered":"2024-09-16T04:54:38.963037+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:38.963037+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29253632,"num_objects":7142,"num_object_clones":0,"num_object_copies":14284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9731,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3677},{"shards":"0,2","objects":502}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12794","reported_seq":14288,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.365114+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:38.365114+0000","last_peered":"2024-09-16T04:54:38.365114+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:38.365114+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29430994,"num_objects":7187,"num_object_clones":0,"num_object_copies":14374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2169,"num_objects_misplaced":506,"num_objects_unfound":0,"num_objects_dirty":7187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12766,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":625,"num_bytes_recovered":2543616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":506},{"shards":"2","objects":4512}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12802","reported_seq":19660,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:38.230369+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:38.230369+0000","last_peered":"2024-09-16T04:54:38.230369+0000","last_clean":"2024-09-16T04:54:38.230369+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:38.230369+0000","last_undegraded":"2024-09-16T04:54:38.230369+0000","last_fullsized":"2024-09-16T04:54:38.230369+0000","mapping_epoch":46,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29392896,"num_objects":7176,"num_object_clones":0,"num_object_copies":14352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9525,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12789","reported_seq":12892,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.328028+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:38.328028+0000","last_peered":"2024-09-16T04:54:38.328028+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:38.328028+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29028969,"num_objects":7088,"num_object_clones":0,"num_object_copies":14176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12788,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":557},{"shards":"2","objects":3681}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13233","reported_seq":13341,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.324009+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:38.324009+0000","last_peered":"2024-09-16T04:54:38.324009+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:38.324009+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29898555,"num_objects":7302,"num_object_clones":0,"num_object_copies":14604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13230,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":551},{"shards":"2","objects":3788}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'12853","reported_seq":19652,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:37.520509+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:37.520509+0000","last_peered":"2024-09-16T04:54:37.520509+0000","last_clean":"2024-09-16T04:54:37.520509+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:37.520509+0000","last_undegraded":"2024-09-16T04:54:37.520509+0000","last_fullsized":"2024-09-16T04:54:37.520509+0000","mapping_epoch":48,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30085120,"num_objects":7345,"num_object_clones":0,"num_object_copies":14690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9607,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13223","reported_seq":13368,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:38.936532+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:38.936532+0000","last_peered":"2024-09-16T04:54:38.936532+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:38.936532+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29463145,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9992,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3670},{"shards":"0,2","objects":515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":470774007,"num_objects":114942,"num_object_clones":0,"num_object_copies":229884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32931,"num_objects_misplaced":506,"num_objects_unfound":0,"num_objects_dirty":114942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171688,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7305,"num_bytes_recovered":29872128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1087004672,"data_stored":1086930427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717816,"kb_used_data":370980,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93654024,"statfs":{"total":96636764160,"available":95901720576,"internally_reserved":0,"allocated":379883520,"data_stored":379600194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593789,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630712,"kb_used_data":324136,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93741128,"statfs":{"total":96636764160,"available":95990915072,"internally_reserved":0,"allocated":331915264,"data_stored":331576274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,2,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711012,"kb_used_data":368396,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93660828,"statfs":{"total":96636764160,"available":95908687872,"internally_reserved":0,"allocated":377237504,"data_stored":376958250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":376729600,"data_stored":376726121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331382784,"data_stored":331319569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378892288,"data_stored":378884737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:44.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:54:44.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'12729", 'reported_seq': 12813, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:38.766470+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:54:38.766470+0000', 'last_peered': '2024-09-16T04:54:38.766470+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:54:38.766470+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28708864, 'num_objects': 7009, 'num_object_clones': 0, 'num_object_copies': 14018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7009, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12729, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3668}, {'shards': '0,2', 'objects': 480}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:44.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:54:44.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'12736", 'reported_seq': 12880, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:38.249328+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:54:38.249328+0000', 'last_peered': '2024-09-16T04:54:38.249328+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:54:38.249328+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29167640, 'num_objects': 7122, 'num_object_clones': 0, 'num_object_copies': 14244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7122, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9475, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 514}, {'shards': '2', 'objects': 3803}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:44.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:54:44.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'13064", 'reported_seq': 13202, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:38.904893+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:54:38.904893+0000', 'last_peered': '2024-09-16T04:54:38.904893+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:54:38.904893+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29691904, 'num_objects': 7249, 'num_object_clones': 0, 'num_object_copies': 14498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7249, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10057, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3793}, {'shards': '0,2', 'objects': 551}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:44.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:54:44.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'12845", 'reported_seq': 12998, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:38.993454+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:54:38.993454+0000', 'last_peered': '2024-09-16T04:54:38.993454+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:54:38.993454+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28733440, 'num_objects': 7015, 'num_object_clones': 0, 'num_object_copies': 14030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7015, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9500, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3616}, {'shards': '0,2', 'objects': 487}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:44.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:54:44.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'12864", 'reported_seq': 13097, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:38.235793+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:54:38.235793+0000', 'last_peered': '2024-09-16T04:54:38.235793+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:54:38.235793+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29560832, 'num_objects': 7217, 'num_object_clones': 0, 'num_object_copies': 14434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7217, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9615, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 514}, {'shards': '2', 'objects': 3896}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:44.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:54:44.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'13076", 'reported_seq': 13211, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:38.963037+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:54:38.963037+0000', 'last_peered': '2024-09-16T04:54:38.963037+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:54:38.963037+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29253632, 'num_objects': 7142, 'num_object_clones': 0, 'num_object_copies': 14284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7142, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9731, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3677}, {'shards': '0,2', 'objects': 502}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:44.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:54:44.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'12794", 'reported_seq': 14288, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:38.365114+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:54:38.365114+0000', 'last_peered': '2024-09-16T04:54:38.365114+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:54:38.365114+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29430994, 'num_objects': 7187, 'num_object_clones': 0, 'num_object_copies': 14374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2169, 'num_objects_misplaced': 506, 'num_objects_unfound': 0, 'num_objects_dirty': 7187, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12766, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 625, 'num_bytes_recovered': 2543616, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 506}, {'shards': '2', 'objects': 4512}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:44.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:54:44.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'12789", 'reported_seq': 12892, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:38.328028+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:54:38.328028+0000', 'last_peered': '2024-09-16T04:54:38.328028+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:54:38.328028+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29028969, 'num_objects': 7088, 'num_object_clones': 0, 'num_object_copies': 14176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7088, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12788, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 557}, {'shards': '2', 'objects': 3681}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:44.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:54:44.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'13233", 'reported_seq': 13341, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:38.324009+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:54:38.324009+0000', 'last_peered': '2024-09-16T04:54:38.324009+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:54:38.324009+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29898555, 'num_objects': 7302, 'num_object_clones': 0, 'num_object_copies': 14604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7302, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13230, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 551}, {'shards': '2', 'objects': 3788}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:44.057 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:54:44.058 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'13223", 'reported_seq': 13368, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:38.936532+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:54:38.936532+0000', 'last_peered': '2024-09-16T04:54:38.936532+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:54:38.936532+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29463145, 'num_objects': 7194, 'num_object_clones': 0, 'num_object_copies': 14388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7194, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9992, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3670}, {'shards': '0,2', 'objects': 515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:44.058 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:54:45.126 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:45.126 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32931/229888 objects degraded (14.325%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":432,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":114944,"data_bytes":470774026,"bytes_used":2108968960,"bytes_avail":287801323520,"bytes_total":289910292480,"degraded_objects":32931,"degraded_total":229888,"degraded_ratio":0.14324801642538976,"misplaced_objects":506,"misplaced_total":229888,"misplaced_ratio":0.0022010718262806235,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":58012,"recovering_keys_per_sec":0,"num_objects_recovered":170,"num_bytes_recovered":696320,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":386},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:54:45.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:54:45.203 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:45.930 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:45.932 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:45.990 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-16T04:54:44.409779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":461017354,"num_objects":112562,"num_object_clones":0,"num_object_copies":225124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32994,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":112562,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174072,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7392,"num_bytes_recovered":30224403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160795,"ondisk_log_size":160795,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2029952,"kb_used_data":1050548,"kb_used_omap":12,"kb_used_meta":979315,"kb_avail":281085568,"statfs":{"total":289910292480,"available":287831621632,"internally_reserved":0,"allocated":1075761152,"data_stored":1074859582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1002818561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,9,12,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28741632,"num_objects":-7017,"num_object_clones":0,"num_object_copies":-14034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":194,"num_objects_misplaced":-450,"num_objects_unfound":0,"num_objects_dirty":-7017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7017,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":256,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002886"},"pg_stats":[{"pgid":"2.f","version":"49'12869","reported_seq":12953,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.942782+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:43.942782+0000","last_peered":"2024-09-16T04:54:43.942782+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:43.942782+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28135424,"num_objects":6869,"num_object_clones":0,"num_object_copies":13738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12869,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3528},{"shards":"0,2","objects":340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13013","reported_seq":13134,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:43.358016+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:43.358016+0000","last_peered":"2024-09-16T04:54:43.358016+0000","last_clean":"2024-09-16T04:54:43.358016+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:43.358016+0000","last_undegraded":"2024-09-16T04:54:43.358016+0000","last_fullsized":"2024-09-16T04:54:43.358016+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28594176,"num_objects":6981,"num_object_clones":0,"num_object_copies":13962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9982,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12870","reported_seq":13014,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.308109+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:43.308109+0000","last_peered":"2024-09-16T04:54:43.308109+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:43.308109+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28618776,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9609,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":380},{"shards":"2","objects":3669}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13168","reported_seq":13243,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:43.381698+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:43.381698+0000","last_peered":"2024-09-16T04:54:43.381698+0000","last_clean":"2024-09-16T04:54:43.381698+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:43.381698+0000","last_undegraded":"2024-09-16T04:54:43.381698+0000","last_fullsized":"2024-09-16T04:54:43.381698+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29392896,"num_objects":7176,"num_object_clones":0,"num_object_copies":14352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13168,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13073","reported_seq":13297,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:43.991845+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:43.991845+0000","last_peered":"2024-09-16T04:54:43.991845+0000","last_clean":"2024-09-16T04:54:43.991845+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:43.991845+0000","last_undegraded":"2024-09-16T04:54:43.991845+0000","last_fullsized":"2024-09-16T04:54:43.991845+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28962816,"num_objects":7071,"num_object_clones":0,"num_object_copies":14142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10105,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13212","reported_seq":13350,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.926004+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:43.926004+0000","last_peered":"2024-09-16T04:54:43.926004+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:43.926004+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29085696,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3645},{"shards":"0,2","objects":403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12998","reported_seq":13147,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.908944+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:43.908944+0000","last_peered":"2024-09-16T04:54:43.908944+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:43.908944+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28106752,"num_objects":6862,"num_object_clones":0,"num_object_copies":13724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9653,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3463},{"shards":"0,2","objects":334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12957","reported_seq":13178,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:44.015692+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:44.015692+0000","last_peered":"2024-09-16T04:54:44.015692+0000","last_clean":"2024-09-16T04:54:44.015692+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:44.015692+0000","last_undegraded":"2024-09-16T04:54:44.015692+0000","last_fullsized":"2024-09-16T04:54:44.015692+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28913664,"num_objects":7059,"num_object_clones":0,"num_object_copies":14118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10027,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13016","reported_seq":13249,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.386614+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:43.386614+0000","last_peered":"2024-09-16T04:54:43.386614+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:43.386614+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28938240,"num_objects":7065,"num_object_clones":0,"num_object_copies":14130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9767,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":362},{"shards":"2","objects":3744}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13218","reported_seq":13353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.745584+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:43.745584+0000","last_peered":"2024-09-16T04:54:43.745584+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:43.745584+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28672000,"num_objects":7000,"num_object_clones":0,"num_object_copies":14000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3535},{"shards":"0,2","objects":360}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12944","reported_seq":14610,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.416116+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:43.416116+0000","last_peered":"2024-09-16T04:54:43.416116+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:43.416116+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28816594,"num_objects":7037,"num_object_clones":0,"num_object_copies":14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2232,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12916,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":711,"num_bytes_recovered":2895872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":357},{"shards":"2","objects":4448}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12947","reported_seq":19807,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:43.370226+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:43.370226+0000","last_peered":"2024-09-16T04:54:43.370226+0000","last_clean":"2024-09-16T04:54:43.370226+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:43.370226+0000","last_undegraded":"2024-09-16T04:54:43.370226+0000","last_fullsized":"2024-09-16T04:54:43.370226+0000","mapping_epoch":46,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28798976,"num_objects":7031,"num_object_clones":0,"num_object_copies":14062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9670,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12959","reported_seq":13062,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.342318+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:43.342318+0000","last_peered":"2024-09-16T04:54:43.342318+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:43.342318+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28332649,"num_objects":6918,"num_object_clones":0,"num_object_copies":13836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12958,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":387},{"shards":"2","objects":3511}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13375","reported_seq":13483,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.242771+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:43.242771+0000","last_peered":"2024-09-16T04:54:43.242771+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:43.242771+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29316923,"num_objects":7160,"num_object_clones":0,"num_object_copies":14320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13372,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":409},{"shards":"2","objects":3646}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13006","reported_seq":19805,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:42.545838+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:42.545838+0000","last_peered":"2024-09-16T04:54:42.545838+0000","last_clean":"2024-09-16T04:54:42.545838+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:42.545838+0000","last_undegraded":"2024-09-16T04:54:42.545838+0000","last_fullsized":"2024-09-16T04:54:42.545838+0000","mapping_epoch":48,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29458432,"num_objects":7192,"num_object_clones":0,"num_object_copies":14384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9760,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13367","reported_seq":13512,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:44.023639+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:44.023639+0000","last_peered":"2024-09-16T04:54:44.023639+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:44.023639+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873321,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10136,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3526},{"shards":"0,2","objects":371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":461017335,"num_objects":112560,"num_object_clones":0,"num_object_copies":225120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32994,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":112560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174070,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7391,"num_bytes_recovered":30224384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1073672192,"data_stored":1073597947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160792,"ondisk_log_size":160792,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699480,"kb_used_data":365100,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93672360,"statfs":{"total":96636764160,"available":95920496640,"internally_reserved":0,"allocated":373862400,"data_stored":373579074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593790,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621276,"kb_used_data":323020,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93750564,"statfs":{"total":96636764160,"available":96000577536,"internally_reserved":0,"allocated":330772480,"data_stored":330433490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709196,"kb_used_data":362428,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93662644,"statfs":{"total":96636764160,"available":95910547456,"internally_reserved":0,"allocated":371126272,"data_stored":370847018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,2,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":370638848,"data_stored":370635369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":330170368,"data_stored":330107153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":372862976,"data_stored":372855425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:48.993 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:49.803 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:49.867 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-16T04:54:48.410781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":460382474,"num_objects":112407,"num_object_clones":0,"num_object_copies":224814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32994,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":112407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174227,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7392,"num_bytes_recovered":30224403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2032948,"kb_used_data":1049384,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281082572,"statfs":{"total":289910292480,"available":287828553728,"internally_reserved":0,"allocated":1074569216,"data_stored":1073667646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1007078401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,6,15,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19824640,"num_objects":-4840,"num_object_clones":0,"num_object_copies":-9680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":119,"num_objects_misplaced":-293,"num_objects_unfound":0,"num_objects_dirty":-4840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4840,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":712704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002886"},"pg_stats":[{"pgid":"2.f","version":"49'12869","reported_seq":12953,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.942782+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:43.942782+0000","last_peered":"2024-09-16T04:54:43.942782+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:43.942782+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28135424,"num_objects":6869,"num_object_clones":0,"num_object_copies":13738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12869,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3528},{"shards":"0,2","objects":340}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13013","reported_seq":13134,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:43.358016+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:43.358016+0000","last_peered":"2024-09-16T04:54:43.358016+0000","last_clean":"2024-09-16T04:54:43.358016+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:43.358016+0000","last_undegraded":"2024-09-16T04:54:43.358016+0000","last_fullsized":"2024-09-16T04:54:43.358016+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28594176,"num_objects":6981,"num_object_clones":0,"num_object_copies":13962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9982,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'12870","reported_seq":13014,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.308109+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:43.308109+0000","last_peered":"2024-09-16T04:54:43.308109+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:43.308109+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28618776,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9609,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":380},{"shards":"2","objects":3669}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13168","reported_seq":13243,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:43.381698+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:43.381698+0000","last_peered":"2024-09-16T04:54:43.381698+0000","last_clean":"2024-09-16T04:54:43.381698+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:43.381698+0000","last_undegraded":"2024-09-16T04:54:43.381698+0000","last_fullsized":"2024-09-16T04:54:43.381698+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29392896,"num_objects":7176,"num_object_clones":0,"num_object_copies":14352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13168,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13073","reported_seq":13297,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:43.991845+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:43.991845+0000","last_peered":"2024-09-16T04:54:43.991845+0000","last_clean":"2024-09-16T04:54:43.991845+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:43.991845+0000","last_undegraded":"2024-09-16T04:54:43.991845+0000","last_fullsized":"2024-09-16T04:54:43.991845+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28962816,"num_objects":7071,"num_object_clones":0,"num_object_copies":14142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10105,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13212","reported_seq":13350,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.926004+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:43.926004+0000","last_peered":"2024-09-16T04:54:43.926004+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:43.926004+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29085696,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10205,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3645},{"shards":"0,2","objects":403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'12998","reported_seq":13147,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.908944+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:43.908944+0000","last_peered":"2024-09-16T04:54:43.908944+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:43.908944+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28106752,"num_objects":6862,"num_object_clones":0,"num_object_copies":13724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9653,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3463},{"shards":"0,2","objects":334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'12957","reported_seq":13178,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:44.015692+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:44.015692+0000","last_peered":"2024-09-16T04:54:44.015692+0000","last_clean":"2024-09-16T04:54:44.015692+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:44.015692+0000","last_undegraded":"2024-09-16T04:54:44.015692+0000","last_fullsized":"2024-09-16T04:54:44.015692+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28913664,"num_objects":7059,"num_object_clones":0,"num_object_copies":14118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10027,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13016","reported_seq":13249,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.386614+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:43.386614+0000","last_peered":"2024-09-16T04:54:43.386614+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:43.386614+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28938240,"num_objects":7065,"num_object_clones":0,"num_object_copies":14130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9767,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":362},{"shards":"2","objects":3744}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13218","reported_seq":13353,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.745584+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:43.745584+0000","last_peered":"2024-09-16T04:54:43.745584+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:43.745584+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28672000,"num_objects":7000,"num_object_clones":0,"num_object_copies":14000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3535},{"shards":"0,2","objects":360}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'12944","reported_seq":14610,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.416116+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:43.416116+0000","last_peered":"2024-09-16T04:54:43.416116+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:43.416116+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28816594,"num_objects":7037,"num_object_clones":0,"num_object_copies":14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2232,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12916,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":711,"num_bytes_recovered":2895872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":357},{"shards":"2","objects":4448}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'12947","reported_seq":19807,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:43.370226+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:43.370226+0000","last_peered":"2024-09-16T04:54:43.370226+0000","last_clean":"2024-09-16T04:54:43.370226+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:43.370226+0000","last_undegraded":"2024-09-16T04:54:43.370226+0000","last_fullsized":"2024-09-16T04:54:43.370226+0000","mapping_epoch":46,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28798976,"num_objects":7031,"num_object_clones":0,"num_object_copies":14062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9670,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'12959","reported_seq":13062,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.342318+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:43.342318+0000","last_peered":"2024-09-16T04:54:43.342318+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:43.342318+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28332649,"num_objects":6918,"num_object_clones":0,"num_object_copies":13836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12958,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":387},{"shards":"2","objects":3511}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13375","reported_seq":13483,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:43.242771+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:43.242771+0000","last_peered":"2024-09-16T04:54:43.242771+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:43.242771+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29316923,"num_objects":7160,"num_object_clones":0,"num_object_copies":14320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13372,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":409},{"shards":"2","objects":3646}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13161","reported_seq":19960,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:47.683991+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:47.683991+0000","last_peered":"2024-09-16T04:54:47.683991+0000","last_clean":"2024-09-16T04:54:47.683991+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:47.683991+0000","last_undegraded":"2024-09-16T04:54:47.683991+0000","last_fullsized":"2024-09-16T04:54:47.683991+0000","mapping_epoch":48,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28823552,"num_objects":7037,"num_object_clones":0,"num_object_copies":14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9915,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13367","reported_seq":13512,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:44.023639+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:44.023639+0000","last_peered":"2024-09-16T04:54:44.023639+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:44.023639+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873321,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10136,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3526},{"shards":"0,2","objects":371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":460382455,"num_objects":112405,"num_object_clones":0,"num_object_copies":224810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32994,"num_objects_misplaced":357,"num_objects_unfound":0,"num_objects_dirty":112405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174225,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7391,"num_bytes_recovered":30224384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1072472064,"data_stored":1072397819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699480,"kb_used_data":365100,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93672360,"statfs":{"total":96636764160,"available":95920496640,"internally_reserved":0,"allocated":373862400,"data_stored":373579074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593791,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624272,"kb_used_data":321856,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93747568,"statfs":{"total":96636764160,"available":95997509632,"internally_reserved":0,"allocated":329580544,"data_stored":329241554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709196,"kb_used_data":362428,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93662644,"statfs":{"total":96636764160,"available":95910547456,"internally_reserved":0,"allocated":371126272,"data_stored":370847018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,2,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":370638848,"data_stored":370635369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328970240,"data_stored":328907025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":372862976,"data_stored":372855425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:49.869 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:50.767 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:50.768 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:50.795 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-09-16T04:54:50.411222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":451436810,"num_objects":110223,"num_object_clones":0,"num_object_copies":220446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33037,"num_objects_misplaced":213,"num_objects_unfound":0,"num_objects_dirty":110223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176411,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7493,"num_bytes_recovered":30638099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2017300,"kb_used_data":1037924,"kb_used_omap":12,"kb_used_meta":979315,"kb_avail":281098220,"statfs":{"total":289910292480,"available":287844577280,"internally_reserved":0,"allocated":1062834176,"data_stored":1061932606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1002818561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,5,7,11,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28164096,"num_objects":-6876,"num_object_clones":0,"num_object_copies":-13752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":162,"num_objects_misplaced":-437,"num_objects_unfound":0,"num_objects_dirty":-6876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6876,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":275,"num_bytes_recovered":1126400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002883"},"pg_stats":[{"pgid":"2.f","version":"49'13005","reported_seq":13089,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.969314+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:48.969314+0000","last_peered":"2024-09-16T04:54:48.969314+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:48.969314+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27578368,"num_objects":6733,"num_object_clones":0,"num_object_copies":13466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13005,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3392},{"shards":"0,2","objects":204}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13150","reported_seq":13271,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:48.320844+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:48.320844+0000","last_peered":"2024-09-16T04:54:48.320844+0000","last_clean":"2024-09-16T04:54:48.320844+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:48.320844+0000","last_undegraded":"2024-09-16T04:54:48.320844+0000","last_fullsized":"2024-09-16T04:54:48.320844+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28033024,"num_objects":6844,"num_object_clones":0,"num_object_copies":13688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10119,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13011","reported_seq":13155,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.437160+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:48.437160+0000","last_peered":"2024-09-16T04:54:48.437160+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:48.437160+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041240,"num_objects":6847,"num_object_clones":0,"num_object_copies":13694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6847,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9750,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":239},{"shards":"2","objects":3528}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13319","reported_seq":13394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:48.365045+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:48.365045+0000","last_peered":"2024-09-16T04:54:48.365045+0000","last_clean":"2024-09-16T04:54:48.365045+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:48.365045+0000","last_undegraded":"2024-09-16T04:54:48.365045+0000","last_fullsized":"2024-09-16T04:54:48.365045+0000","mapping_epoch":35,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28774400,"num_objects":7025,"num_object_clones":0,"num_object_copies":14050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13319,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13220","reported_seq":13444,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:48.975447+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:48.975447+0000","last_peered":"2024-09-16T04:54:48.975447+0000","last_clean":"2024-09-16T04:54:48.975447+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:48.975447+0000","last_undegraded":"2024-09-16T04:54:48.975447+0000","last_fullsized":"2024-09-16T04:54:48.975447+0000","mapping_epoch":34,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28360704,"num_objects":6924,"num_object_clones":0,"num_object_copies":13848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10252,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13373","reported_seq":13507,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.971846+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:48.971846+0000","last_peered":"2024-09-16T04:54:48.971846+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:48.971846+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28426240,"num_objects":6940,"num_object_clones":0,"num_object_copies":13880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10366,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3484},{"shards":"0,2","objects":242}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13126","reported_seq":13278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:49.023870+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:49.023870+0000","last_peered":"2024-09-16T04:54:49.023870+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:49.023870+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27582464,"num_objects":6734,"num_object_clones":0,"num_object_copies":13468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9781,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3335},{"shards":"0,2","objects":206}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13108","reported_seq":13329,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:48.950286+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:48.950286+0000","last_peered":"2024-09-16T04:54:48.950286+0000","last_clean":"2024-09-16T04:54:48.950286+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:48.950286+0000","last_undegraded":"2024-09-16T04:54:48.950286+0000","last_fullsized":"2024-09-16T04:54:48.950286+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28295168,"num_objects":6908,"num_object_clones":0,"num_object_copies":13816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10178,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13152","reported_seq":13385,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.391341+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:48.391341+0000","last_peered":"2024-09-16T04:54:48.391341+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:48.391341+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28381184,"num_objects":6929,"num_object_clones":0,"num_object_copies":13858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9903,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":226},{"shards":"2","objects":3608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13377","reported_seq":13509,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.963755+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:48.963755+0000","last_peered":"2024-09-16T04:54:48.963755+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:48.963755+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28020736,"num_objects":6841,"num_object_clones":0,"num_object_copies":13682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10032,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3376},{"shards":"0,2","objects":201}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13088","reported_seq":14955,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.436967+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:48.436967+0000","last_peered":"2024-09-16T04:54:48.436967+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:48.436967+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28226770,"num_objects":6893,"num_object_clones":0,"num_object_copies":13786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2275,"num_objects_misplaced":213,"num_objects_unfound":0,"num_objects_dirty":6893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13060,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":812,"num_bytes_recovered":3309568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":213},{"shards":"2","objects":4405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13082","reported_seq":19943,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:48.374606+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:48.374606+0000","last_peered":"2024-09-16T04:54:48.374606+0000","last_clean":"2024-09-16T04:54:48.374606+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:48.374606+0000","last_undegraded":"2024-09-16T04:54:48.374606+0000","last_fullsized":"2024-09-16T04:54:48.374606+0000","mapping_epoch":46,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28246016,"num_objects":6896,"num_object_clones":0,"num_object_copies":13792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9805,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13100","reported_seq":13201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.417377+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:48.417377+0000","last_peered":"2024-09-16T04:54:48.417377+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:48.417377+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27755113,"num_objects":6777,"num_object_clones":0,"num_object_copies":13554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13099,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":246},{"shards":"2","objects":3370}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13540","reported_seq":13648,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.408577+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:48.408577+0000","last_peered":"2024-09-16T04:54:48.408577+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:48.408577+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"42'3500","ondisk_log_start":"42'3500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28641083,"num_objects":6995,"num_object_clones":0,"num_object_copies":13990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13537,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":244},{"shards":"2","objects":3481}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13161","reported_seq":19960,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:47.683991+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:47.683991+0000","last_peered":"2024-09-16T04:54:47.683991+0000","last_clean":"2024-09-16T04:54:47.683991+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:47.683991+0000","last_undegraded":"2024-09-16T04:54:47.683991+0000","last_fullsized":"2024-09-16T04:54:47.683991+0000","mapping_epoch":48,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28823552,"num_objects":7037,"num_object_clones":0,"num_object_copies":14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9915,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13519","reported_seq":13664,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:49.023806+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:49.023806+0000","last_peered":"2024-09-16T04:54:49.023806+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:49.023806+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"42'3500","ondisk_log_start":"42'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28250729,"num_objects":6898,"num_object_clones":0,"num_object_copies":13796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10288,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3374},{"shards":"0,2","objects":219}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":451436791,"num_objects":110221,"num_object_clones":0,"num_object_copies":220442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33037,"num_objects_misplaced":213,"num_objects_unfound":0,"num_objects_dirty":110221,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176409,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7492,"num_bytes_recovered":30638080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1060429824,"data_stored":1060355579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160731,"ondisk_log_size":160731,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697964,"kb_used_data":359452,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93673876,"statfs":{"total":96636764160,"available":95922049024,"internally_reserved":0,"allocated":368078848,"data_stored":367795522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593791,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624272,"kb_used_data":321856,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93747568,"statfs":{"total":96636764160,"available":95997509632,"internally_reserved":0,"allocated":329580544,"data_stored":329241554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695064,"kb_used_data":356616,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93676776,"statfs":{"total":96636764160,"available":95925018624,"internally_reserved":0,"allocated":365174784,"data_stored":364895530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,4,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364576768,"data_stored":364573289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328970240,"data_stored":328907025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":366882816,"data_stored":366875265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:50.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:54:50.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'13005", 'reported_seq': 13089, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:48.969314+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:54:48.969314+0000', 'last_peered': '2024-09-16T04:54:48.969314+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:54:48.969314+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27578368, 'num_objects': 6733, 'num_object_clones': 0, 'num_object_copies': 13466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6733, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13005, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3392}, {'shards': '0,2', 'objects': 204}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:50.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:54:50.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'13011", 'reported_seq': 13155, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:48.437160+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:54:48.437160+0000', 'last_peered': '2024-09-16T04:54:48.437160+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:54:48.437160+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28041240, 'num_objects': 6847, 'num_object_clones': 0, 'num_object_copies': 13694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6847, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9750, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 239}, {'shards': '2', 'objects': 3528}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:50.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:54:50.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'13373", 'reported_seq': 13507, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:48.971846+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:54:48.971846+0000', 'last_peered': '2024-09-16T04:54:48.971846+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:54:48.971846+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "39'3300", 'ondisk_log_start': "39'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28426240, 'num_objects': 6940, 'num_object_clones': 0, 'num_object_copies': 13880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6940, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10366, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3484}, {'shards': '0,2', 'objects': 242}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:50.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:54:50.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'13126", 'reported_seq': 13278, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:49.023870+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:54:49.023870+0000', 'last_peered': '2024-09-16T04:54:49.023870+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:54:49.023870+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27582464, 'num_objects': 6734, 'num_object_clones': 0, 'num_object_copies': 13468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6734, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9781, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3335}, {'shards': '0,2', 'objects': 206}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:50.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:54:50.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'13152", 'reported_seq': 13385, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:48.391341+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:54:48.391341+0000', 'last_peered': '2024-09-16T04:54:48.391341+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:54:48.391341+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28381184, 'num_objects': 6929, 'num_object_clones': 0, 'num_object_copies': 13858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6929, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9903, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 226}, {'shards': '2', 'objects': 3608}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:50.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:54:50.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'13377", 'reported_seq': 13509, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:48.963755+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:54:48.963755+0000', 'last_peered': '2024-09-16T04:54:48.963755+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:54:48.963755+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "39'3300", 'ondisk_log_start': "39'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28020736, 'num_objects': 6841, 'num_object_clones': 0, 'num_object_copies': 13682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6841, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10032, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3376}, {'shards': '0,2', 'objects': 201}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:50.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:54:50.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'13088", 'reported_seq': 14955, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:48.436967+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:54:48.436967+0000', 'last_peered': '2024-09-16T04:54:48.436967+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:54:48.436967+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28226770, 'num_objects': 6893, 'num_object_clones': 0, 'num_object_copies': 13786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2275, 'num_objects_misplaced': 213, 'num_objects_unfound': 0, 'num_objects_dirty': 6893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13060, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 812, 'num_bytes_recovered': 3309568, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 213}, {'shards': '2', 'objects': 4405}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:50.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:54:50.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'13100", 'reported_seq': 13201, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:48.417377+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:54:48.417377+0000', 'last_peered': '2024-09-16T04:54:48.417377+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:54:48.417377+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27755113, 'num_objects': 6777, 'num_object_clones': 0, 'num_object_copies': 13554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13099, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 246}, {'shards': '2', 'objects': 3370}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:50.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:54:50.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'13540", 'reported_seq': 13648, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:48.408577+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:54:48.408577+0000', 'last_peered': '2024-09-16T04:54:48.408577+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:54:48.408577+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "42'3500", 'ondisk_log_start': "42'3500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28641083, 'num_objects': 6995, 'num_object_clones': 0, 'num_object_copies': 13990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6995, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13537, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 244}, {'shards': '2', 'objects': 3481}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:50.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:54:50.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'13519", 'reported_seq': 13664, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:49.023806+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:54:49.023806+0000', 'last_peered': '2024-09-16T04:54:49.023806+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:54:49.023806+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "42'3500", 'ondisk_log_start': "42'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28250729, 'num_objects': 6898, 'num_object_clones': 0, 'num_object_copies': 13796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6898, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10288, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3374}, {'shards': '0,2', 'objects': 219}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:50.799 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:54:51.799 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:51.799 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33037/220446 objects degraded (14.986%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":439,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":110223,"data_bytes":451436810,"bytes_used":2065715200,"bytes_avail":287844577280,"bytes_total":289910292480,"degraded_objects":33037,"degraded_total":220446,"degraded_ratio":0.14986436587645047,"misplaced_objects":213,"misplaced_total":220446,"misplaced_ratio":0.0009662230206036852,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":93844,"recovering_keys_per_sec":0,"num_objects_recovered":275,"num_bytes_recovered":1126400,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":572},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:54:51.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:54:51.879 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:52.566 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:52.568 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:52.621 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-16T04:54:52.411644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":451436810,"num_objects":110223,"num_object_clones":0,"num_object_copies":220446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33037,"num_objects_misplaced":213,"num_objects_unfound":0,"num_objects_dirty":110223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176411,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7493,"num_bytes_recovered":30638099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2017300,"kb_used_data":1037924,"kb_used_omap":12,"kb_used_meta":979315,"kb_avail":281098220,"statfs":{"total":289910292480,"available":287844577280,"internally_reserved":0,"allocated":1062834176,"data_stored":1061932606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1002818561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,5,7,11,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19337216,"num_objects":-4721,"num_object_clones":0,"num_object_copies":-9442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":106,"num_objects_misplaced":-293,"num_objects_unfound":0,"num_objects_dirty":-4721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4721,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":765952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002839"},"pg_stats":[{"pgid":"2.f","version":"49'13005","reported_seq":13089,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.969314+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:48.969314+0000","last_peered":"2024-09-16T04:54:48.969314+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:48.969314+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27578368,"num_objects":6733,"num_object_clones":0,"num_object_copies":13466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13005,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3392},{"shards":"0,2","objects":204}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13150","reported_seq":13271,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:48.320844+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:48.320844+0000","last_peered":"2024-09-16T04:54:48.320844+0000","last_clean":"2024-09-16T04:54:48.320844+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:48.320844+0000","last_undegraded":"2024-09-16T04:54:48.320844+0000","last_fullsized":"2024-09-16T04:54:48.320844+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28033024,"num_objects":6844,"num_object_clones":0,"num_object_copies":13688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10119,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13011","reported_seq":13155,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.437160+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:48.437160+0000","last_peered":"2024-09-16T04:54:48.437160+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:48.437160+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28041240,"num_objects":6847,"num_object_clones":0,"num_object_copies":13694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6847,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9750,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":239},{"shards":"2","objects":3528}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13319","reported_seq":13394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:48.365045+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:48.365045+0000","last_peered":"2024-09-16T04:54:48.365045+0000","last_clean":"2024-09-16T04:54:48.365045+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:48.365045+0000","last_undegraded":"2024-09-16T04:54:48.365045+0000","last_fullsized":"2024-09-16T04:54:48.365045+0000","mapping_epoch":35,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28774400,"num_objects":7025,"num_object_clones":0,"num_object_copies":14050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13319,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13220","reported_seq":13444,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:48.975447+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:48.975447+0000","last_peered":"2024-09-16T04:54:48.975447+0000","last_clean":"2024-09-16T04:54:48.975447+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:48.975447+0000","last_undegraded":"2024-09-16T04:54:48.975447+0000","last_fullsized":"2024-09-16T04:54:48.975447+0000","mapping_epoch":34,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28360704,"num_objects":6924,"num_object_clones":0,"num_object_copies":13848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10252,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13373","reported_seq":13507,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.971846+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:48.971846+0000","last_peered":"2024-09-16T04:54:48.971846+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:48.971846+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28426240,"num_objects":6940,"num_object_clones":0,"num_object_copies":13880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10366,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3484},{"shards":"0,2","objects":242}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13126","reported_seq":13278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:49.023870+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:49.023870+0000","last_peered":"2024-09-16T04:54:49.023870+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:49.023870+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27582464,"num_objects":6734,"num_object_clones":0,"num_object_copies":13468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9781,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3335},{"shards":"0,2","objects":206}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13108","reported_seq":13329,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:48.950286+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:48.950286+0000","last_peered":"2024-09-16T04:54:48.950286+0000","last_clean":"2024-09-16T04:54:48.950286+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:48.950286+0000","last_undegraded":"2024-09-16T04:54:48.950286+0000","last_fullsized":"2024-09-16T04:54:48.950286+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28295168,"num_objects":6908,"num_object_clones":0,"num_object_copies":13816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10178,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13152","reported_seq":13385,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.391341+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:48.391341+0000","last_peered":"2024-09-16T04:54:48.391341+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:48.391341+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28381184,"num_objects":6929,"num_object_clones":0,"num_object_copies":13858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9903,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":226},{"shards":"2","objects":3608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13377","reported_seq":13509,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.963755+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:48.963755+0000","last_peered":"2024-09-16T04:54:48.963755+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:48.963755+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28020736,"num_objects":6841,"num_object_clones":0,"num_object_copies":13682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10032,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3376},{"shards":"0,2","objects":201}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13088","reported_seq":14955,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.436967+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:48.436967+0000","last_peered":"2024-09-16T04:54:48.436967+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:48.436967+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28226770,"num_objects":6893,"num_object_clones":0,"num_object_copies":13786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2275,"num_objects_misplaced":213,"num_objects_unfound":0,"num_objects_dirty":6893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13060,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":812,"num_bytes_recovered":3309568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":213},{"shards":"2","objects":4405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13082","reported_seq":19943,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:48.374606+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:48.374606+0000","last_peered":"2024-09-16T04:54:48.374606+0000","last_clean":"2024-09-16T04:54:48.374606+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:48.374606+0000","last_undegraded":"2024-09-16T04:54:48.374606+0000","last_fullsized":"2024-09-16T04:54:48.374606+0000","mapping_epoch":46,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28246016,"num_objects":6896,"num_object_clones":0,"num_object_copies":13792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9805,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13100","reported_seq":13201,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.417377+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:48.417377+0000","last_peered":"2024-09-16T04:54:48.417377+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:48.417377+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27755113,"num_objects":6777,"num_object_clones":0,"num_object_copies":13554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13099,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":246},{"shards":"2","objects":3370}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13540","reported_seq":13648,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:48.408577+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:48.408577+0000","last_peered":"2024-09-16T04:54:48.408577+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:48.408577+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"42'3500","ondisk_log_start":"42'3500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28641083,"num_objects":6995,"num_object_clones":0,"num_object_copies":13990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13537,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":244},{"shards":"2","objects":3481}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13161","reported_seq":19960,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:47.683991+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:47.683991+0000","last_peered":"2024-09-16T04:54:47.683991+0000","last_clean":"2024-09-16T04:54:47.683991+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:47.683991+0000","last_undegraded":"2024-09-16T04:54:47.683991+0000","last_fullsized":"2024-09-16T04:54:47.683991+0000","mapping_epoch":48,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28823552,"num_objects":7037,"num_object_clones":0,"num_object_copies":14074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9915,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13519","reported_seq":13664,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:49.023806+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:49.023806+0000","last_peered":"2024-09-16T04:54:49.023806+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:49.023806+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"42'3500","ondisk_log_start":"42'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28250729,"num_objects":6898,"num_object_clones":0,"num_object_copies":13796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10288,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3374},{"shards":"0,2","objects":219}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":451436791,"num_objects":110221,"num_object_clones":0,"num_object_copies":220442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33037,"num_objects_misplaced":213,"num_objects_unfound":0,"num_objects_dirty":110221,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":176409,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7492,"num_bytes_recovered":30638080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1060429824,"data_stored":1060355579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160731,"ondisk_log_size":160731,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697964,"kb_used_data":359452,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93673876,"statfs":{"total":96636764160,"available":95922049024,"internally_reserved":0,"allocated":368078848,"data_stored":367795522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593791,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624272,"kb_used_data":321856,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93747568,"statfs":{"total":96636764160,"available":95997509632,"internally_reserved":0,"allocated":329580544,"data_stored":329241554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695064,"kb_used_data":356616,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93676776,"statfs":{"total":96636764160,"available":95925018624,"internally_reserved":0,"allocated":365174784,"data_stored":364895530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,4,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364576768,"data_stored":364573289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":328970240,"data_stored":328907025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":366882816,"data_stored":366875265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:55.623 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:56.507 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:56.509 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:56.588 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-09-16T04:54:56.412546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":442122506,"num_objects":107949,"num_object_clones":0,"num_object_copies":215898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33078,"num_objects_misplaced":71,"num_objects_unfound":0,"num_objects_dirty":107949,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178685,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7594,"num_bytes_recovered":31051795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2021500,"kb_used_data":1025476,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":281094020,"statfs":{"total":289910292480,"available":287840276480,"internally_reserved":0,"allocated":1050087424,"data_stored":1049185854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1019857921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,2,6,8,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18894848,"num_objects":-4613,"num_object_clones":0,"num_object_copies":-9226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":84,"num_objects_misplaced":-286,"num_objects_unfound":0,"num_objects_dirty":-4613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4613,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":827392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002767"},"pg_stats":[{"pgid":"2.f","version":"49'13132","reported_seq":13216,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.951637+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:53.951637+0000","last_peered":"2024-09-16T04:54:53.951637+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:53.951637+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27058176,"num_objects":6606,"num_object_clones":0,"num_object_copies":13212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13132,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3265},{"shards":"0,2","objects":77}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13276","reported_seq":13397,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:53.513357+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:53.513357+0000","last_peered":"2024-09-16T04:54:53.513357+0000","last_clean":"2024-09-16T04:54:53.513357+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:53.513357+0000","last_undegraded":"2024-09-16T04:54:53.513357+0000","last_fullsized":"2024-09-16T04:54:53.513357+0000","mapping_epoch":35,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27516928,"num_objects":6718,"num_object_clones":0,"num_object_copies":13436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10245,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13164","reported_seq":13307,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.430548+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:53.430548+0000","last_peered":"2024-09-16T04:54:53.430548+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:53.430548+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27414552,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9903,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":86},{"shards":"2","objects":3375}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13479","reported_seq":13554,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:53.427353+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:53.427353+0000","last_peered":"2024-09-16T04:54:53.427353+0000","last_clean":"2024-09-16T04:54:53.427353+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:53.427353+0000","last_undegraded":"2024-09-16T04:54:53.427353+0000","last_fullsized":"2024-09-16T04:54:53.427353+0000","mapping_epoch":35,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28119040,"num_objects":6865,"num_object_clones":0,"num_object_copies":13730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13479,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13369","reported_seq":13593,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:53.975083+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:53.975083+0000","last_peered":"2024-09-16T04:54:53.975083+0000","last_clean":"2024-09-16T04:54:53.975083+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:53.975083+0000","last_undegraded":"2024-09-16T04:54:53.975083+0000","last_fullsized":"2024-09-16T04:54:53.975083+0000","mapping_epoch":34,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10401,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13525","reported_seq":13663,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:54.037310+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:54.037310+0000","last_peered":"2024-09-16T04:54:54.037310+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:54.037310+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27803648,"num_objects":6788,"num_object_clones":0,"num_object_copies":13576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10518,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3332},{"shards":"0,2","objects":90}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13264","reported_seq":13415,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:54.067896+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:54.067896+0000","last_peered":"2024-09-16T04:54:54.067896+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:54.067896+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27017216,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9919,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3197},{"shards":"0,2","objects":68}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13246","reported_seq":13466,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:54.037110+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:54.037110+0000","last_peered":"2024-09-16T04:54:54.037110+0000","last_clean":"2024-09-16T04:54:54.037110+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:54.037110+0000","last_undegraded":"2024-09-16T04:54:54.037110+0000","last_fullsized":"2024-09-16T04:54:54.037110+0000","mapping_epoch":34,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13289","reported_seq":13522,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.341676+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:53.341676+0000","last_peered":"2024-09-16T04:54:53.341676+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:53.341676+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27820032,"num_objects":6792,"num_object_clones":0,"num_object_copies":13584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10040,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":89},{"shards":"2","objects":3471}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13504","reported_seq":13639,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:54.022920+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:54.022920+0000","last_peered":"2024-09-16T04:54:54.022920+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:54.022920+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27500544,"num_objects":6714,"num_object_clones":0,"num_object_copies":13428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10159,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3249},{"shards":"0,2","objects":74}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13225","reported_seq":15295,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.499903+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:53.499903+0000","last_peered":"2024-09-16T04:54:53.499903+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:53.499903+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27665618,"num_objects":6756,"num_object_clones":0,"num_object_copies":13512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2316,"num_objects_misplaced":71,"num_objects_unfound":0,"num_objects_dirty":6756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13197,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":913,"num_bytes_recovered":3723264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":71},{"shards":"2","objects":4369}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13234","reported_seq":20096,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:53.486637+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:53.486637+0000","last_peered":"2024-09-16T04:54:53.486637+0000","last_clean":"2024-09-16T04:54:53.486637+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:53.486637+0000","last_undegraded":"2024-09-16T04:54:53.486637+0000","last_fullsized":"2024-09-16T04:54:53.486637+0000","mapping_epoch":46,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9957,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13266","reported_seq":13368,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.477357+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:53.477357+0000","last_peered":"2024-09-16T04:54:53.477357+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:53.477357+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27075177,"num_objects":6611,"num_object_clones":0,"num_object_copies":13222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13265,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":80},{"shards":"2","objects":3204}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13681","reported_seq":13789,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.337958+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:53.337958+0000","last_peered":"2024-09-16T04:54:53.337958+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:53.337958+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28063547,"num_objects":6854,"num_object_clones":0,"num_object_copies":13708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13678,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":103},{"shards":"2","objects":3340}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13296","reported_seq":20095,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:52.673988+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:52.673988+0000","last_peered":"2024-09-16T04:54:52.673988+0000","last_clean":"2024-09-16T04:54:52.673988+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:52.673988+0000","last_undegraded":"2024-09-16T04:54:52.673988+0000","last_fullsized":"2024-09-16T04:54:52.673988+0000","mapping_epoch":48,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28270592,"num_objects":6902,"num_object_clones":0,"num_object_copies":13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10050,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13655","reported_seq":13800,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:54.034175+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:54.034175+0000","last_peered":"2024-09-16T04:54:54.034175+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:54.034175+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27693673,"num_objects":6762,"num_object_clones":0,"num_object_copies":13524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10424,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3238},{"shards":"0,2","objects":83}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":442122487,"num_objects":107947,"num_object_clones":0,"num_object_copies":215894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33078,"num_objects_misplaced":71,"num_objects_unfound":0,"num_objects_dirty":107947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178683,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7593,"num_bytes_recovered":31051776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1047560192,"data_stored":1047485947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160905,"ondisk_log_size":160905,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696392,"kb_used_data":353716,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93675448,"statfs":{"total":96636764160,"available":95923658752,"internally_reserved":0,"allocated":362205184,"data_stored":361921858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593792,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631496,"kb_used_data":320760,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93740344,"statfs":{"total":96636764160,"available":95990112256,"internally_reserved":0,"allocated":328458240,"data_stored":328119250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693612,"kb_used_data":351000,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93678228,"statfs":{"total":96636764160,"available":95926505472,"internally_reserved":0,"allocated":359424000,"data_stored":359144746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":358875136,"data_stored":358871657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327823360,"data_stored":327760145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360861696,"data_stored":360854145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:56.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:57.371 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:57.373 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:57.423 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-09-16T04:54:56.412546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":442122506,"num_objects":107949,"num_object_clones":0,"num_object_copies":215898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33078,"num_objects_misplaced":71,"num_objects_unfound":0,"num_objects_dirty":107949,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178685,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7594,"num_bytes_recovered":31051795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2021500,"kb_used_data":1025476,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":281094020,"statfs":{"total":289910292480,"available":287840276480,"internally_reserved":0,"allocated":1050087424,"data_stored":1049185854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1019857921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,2,6,8,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18894848,"num_objects":-4613,"num_object_clones":0,"num_object_copies":-9226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":84,"num_objects_misplaced":-286,"num_objects_unfound":0,"num_objects_dirty":-4613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4613,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":827392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002767"},"pg_stats":[{"pgid":"2.f","version":"49'13132","reported_seq":13216,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.951637+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:53.951637+0000","last_peered":"2024-09-16T04:54:53.951637+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:53.951637+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27058176,"num_objects":6606,"num_object_clones":0,"num_object_copies":13212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13132,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3265},{"shards":"0,2","objects":77}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13276","reported_seq":13397,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:53.513357+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:53.513357+0000","last_peered":"2024-09-16T04:54:53.513357+0000","last_clean":"2024-09-16T04:54:53.513357+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:53.513357+0000","last_undegraded":"2024-09-16T04:54:53.513357+0000","last_fullsized":"2024-09-16T04:54:53.513357+0000","mapping_epoch":35,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27516928,"num_objects":6718,"num_object_clones":0,"num_object_copies":13436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10245,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13164","reported_seq":13307,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.430548+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:53.430548+0000","last_peered":"2024-09-16T04:54:53.430548+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:53.430548+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27414552,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9903,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":86},{"shards":"2","objects":3375}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13479","reported_seq":13554,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:53.427353+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:53.427353+0000","last_peered":"2024-09-16T04:54:53.427353+0000","last_clean":"2024-09-16T04:54:53.427353+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:53.427353+0000","last_undegraded":"2024-09-16T04:54:53.427353+0000","last_fullsized":"2024-09-16T04:54:53.427353+0000","mapping_epoch":35,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28119040,"num_objects":6865,"num_object_clones":0,"num_object_copies":13730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13479,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13369","reported_seq":13593,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:53.975083+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:53.975083+0000","last_peered":"2024-09-16T04:54:53.975083+0000","last_clean":"2024-09-16T04:54:53.975083+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:53.975083+0000","last_undegraded":"2024-09-16T04:54:53.975083+0000","last_fullsized":"2024-09-16T04:54:53.975083+0000","mapping_epoch":34,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10401,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13525","reported_seq":13663,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:54.037310+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:54.037310+0000","last_peered":"2024-09-16T04:54:54.037310+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:54.037310+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27803648,"num_objects":6788,"num_object_clones":0,"num_object_copies":13576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10518,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3332},{"shards":"0,2","objects":90}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13264","reported_seq":13415,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:54.067896+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:54.067896+0000","last_peered":"2024-09-16T04:54:54.067896+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:54.067896+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27017216,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9919,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3197},{"shards":"0,2","objects":68}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13246","reported_seq":13466,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:54.037110+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:54.037110+0000","last_peered":"2024-09-16T04:54:54.037110+0000","last_clean":"2024-09-16T04:54:54.037110+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:54.037110+0000","last_undegraded":"2024-09-16T04:54:54.037110+0000","last_fullsized":"2024-09-16T04:54:54.037110+0000","mapping_epoch":34,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13289","reported_seq":13522,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.341676+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:53.341676+0000","last_peered":"2024-09-16T04:54:53.341676+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:53.341676+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27820032,"num_objects":6792,"num_object_clones":0,"num_object_copies":13584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10040,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":89},{"shards":"2","objects":3471}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13504","reported_seq":13639,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:54.022920+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:54.022920+0000","last_peered":"2024-09-16T04:54:54.022920+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:54.022920+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27500544,"num_objects":6714,"num_object_clones":0,"num_object_copies":13428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10159,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3249},{"shards":"0,2","objects":74}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13225","reported_seq":15295,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.499903+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:53.499903+0000","last_peered":"2024-09-16T04:54:53.499903+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:53.499903+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27665618,"num_objects":6756,"num_object_clones":0,"num_object_copies":13512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2316,"num_objects_misplaced":71,"num_objects_unfound":0,"num_objects_dirty":6756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13197,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":913,"num_bytes_recovered":3723264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":71},{"shards":"2","objects":4369}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13234","reported_seq":20096,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:53.486637+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:53.486637+0000","last_peered":"2024-09-16T04:54:53.486637+0000","last_clean":"2024-09-16T04:54:53.486637+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:53.486637+0000","last_undegraded":"2024-09-16T04:54:53.486637+0000","last_fullsized":"2024-09-16T04:54:53.486637+0000","mapping_epoch":46,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9957,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13266","reported_seq":13368,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.477357+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:53.477357+0000","last_peered":"2024-09-16T04:54:53.477357+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:53.477357+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27075177,"num_objects":6611,"num_object_clones":0,"num_object_copies":13222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13265,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":80},{"shards":"2","objects":3204}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13681","reported_seq":13789,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.337958+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:53.337958+0000","last_peered":"2024-09-16T04:54:53.337958+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:53.337958+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28063547,"num_objects":6854,"num_object_clones":0,"num_object_copies":13708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13678,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":103},{"shards":"2","objects":3340}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13296","reported_seq":20095,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:52.673988+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:52.673988+0000","last_peered":"2024-09-16T04:54:52.673988+0000","last_clean":"2024-09-16T04:54:52.673988+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:52.673988+0000","last_undegraded":"2024-09-16T04:54:52.673988+0000","last_fullsized":"2024-09-16T04:54:52.673988+0000","mapping_epoch":48,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28270592,"num_objects":6902,"num_object_clones":0,"num_object_copies":13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10050,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13655","reported_seq":13800,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:54.034175+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:54.034175+0000","last_peered":"2024-09-16T04:54:54.034175+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:54.034175+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27693673,"num_objects":6762,"num_object_clones":0,"num_object_copies":13524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10424,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3238},{"shards":"0,2","objects":83}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":442122487,"num_objects":107947,"num_object_clones":0,"num_object_copies":215894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33078,"num_objects_misplaced":71,"num_objects_unfound":0,"num_objects_dirty":107947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178683,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7593,"num_bytes_recovered":31051776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1047560192,"data_stored":1047485947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160905,"ondisk_log_size":160905,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696392,"kb_used_data":353716,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93675448,"statfs":{"total":96636764160,"available":95923658752,"internally_reserved":0,"allocated":362205184,"data_stored":361921858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593792,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631496,"kb_used_data":320760,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93740344,"statfs":{"total":96636764160,"available":95990112256,"internally_reserved":0,"allocated":328458240,"data_stored":328119250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693612,"kb_used_data":351000,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93678228,"statfs":{"total":96636764160,"available":95926505472,"internally_reserved":0,"allocated":359424000,"data_stored":359144746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":358875136,"data_stored":358871657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":327823360,"data_stored":327760145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360861696,"data_stored":360854145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:54:57.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:54:57.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'13132", 'reported_seq': 13216, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:53.951637+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:54:53.951637+0000', 'last_peered': '2024-09-16T04:54:53.951637+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:54:53.951637+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27058176, 'num_objects': 6606, 'num_object_clones': 0, 'num_object_copies': 13212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6606, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13132, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3265}, {'shards': '0,2', 'objects': 77}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:57.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:54:57.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'13164", 'reported_seq': 13307, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:53.430548+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:54:53.430548+0000', 'last_peered': '2024-09-16T04:54:53.430548+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:54:53.430548+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27414552, 'num_objects': 6694, 'num_object_clones': 0, 'num_object_copies': 13388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6694, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9903, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 86}, {'shards': '2', 'objects': 3375}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:57.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:54:57.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'13525", 'reported_seq': 13663, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:54.037310+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:54:54.037310+0000', 'last_peered': '2024-09-16T04:54:54.037310+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:54:54.037310+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'3500", 'ondisk_log_start': "45'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27803648, 'num_objects': 6788, 'num_object_clones': 0, 'num_object_copies': 13576, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6788, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10518, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3332}, {'shards': '0,2', 'objects': 90}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:57.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:54:57.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'13264", 'reported_seq': 13415, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:54.067896+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:54:54.067896+0000', 'last_peered': '2024-09-16T04:54:54.067896+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:54:54.067896+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27017216, 'num_objects': 6596, 'num_object_clones': 0, 'num_object_copies': 13192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6596, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9919, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3197}, {'shards': '0,2', 'objects': 68}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:57.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:54:57.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'13289", 'reported_seq': 13522, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:53.341676+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:54:53.341676+0000', 'last_peered': '2024-09-16T04:54:53.341676+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:54:53.341676+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "39'3200", 'ondisk_log_start': "39'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27820032, 'num_objects': 6792, 'num_object_clones': 0, 'num_object_copies': 13584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6792, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10040, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 89}, {'shards': '2', 'objects': 3471}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:57.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:54:57.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'13504", 'reported_seq': 13639, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:54.022920+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:54:54.022920+0000', 'last_peered': '2024-09-16T04:54:54.022920+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:54:54.022920+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'3500", 'ondisk_log_start': "45'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27500544, 'num_objects': 6714, 'num_object_clones': 0, 'num_object_copies': 13428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6714, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10159, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3249}, {'shards': '0,2', 'objects': 74}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:57.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:54:57.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'13225", 'reported_seq': 15295, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:53.499903+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:54:53.499903+0000', 'last_peered': '2024-09-16T04:54:53.499903+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:54:53.499903+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "39'3200", 'ondisk_log_start': "39'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27665618, 'num_objects': 6756, 'num_object_clones': 0, 'num_object_copies': 13512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2316, 'num_objects_misplaced': 71, 'num_objects_unfound': 0, 'num_objects_dirty': 6756, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13197, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 913, 'num_bytes_recovered': 3723264, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 71}, {'shards': '2', 'objects': 4369}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:57.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:54:57.427 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'13266", 'reported_seq': 13368, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:53.477357+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:54:53.477357+0000', 'last_peered': '2024-09-16T04:54:53.477357+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:54:53.477357+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "39'3200", 'ondisk_log_start': "39'3200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27075177, 'num_objects': 6611, 'num_object_clones': 0, 'num_object_copies': 13222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6611, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13265, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 80}, {'shards': '2', 'objects': 3204}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:57.428 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:54:57.428 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'13681", 'reported_seq': 13789, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:53.337958+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:54:53.337958+0000', 'last_peered': '2024-09-16T04:54:53.337958+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:54:53.337958+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'3600", 'ondisk_log_start': "45'3600", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28063547, 'num_objects': 6854, 'num_object_clones': 0, 'num_object_copies': 13708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6854, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13678, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 103}, {'shards': '2', 'objects': 3340}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:54:57.428 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:54:57.428 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'13655", 'reported_seq': 13800, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:54.034175+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:54:54.034175+0000', 'last_peered': '2024-09-16T04:54:54.034175+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:54:54.034175+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'3600", 'ondisk_log_start': "45'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27693673, 'num_objects': 6762, 'num_object_clones': 0, 'num_object_copies': 13524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10424, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3238}, {'shards': '0,2', 'objects': 83}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:54:57.428 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:54:58.319 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:58.319 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33078/215898 objects degraded (15.321%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":446,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":107949,"data_bytes":442122506,"bytes_used":2070016000,"bytes_avail":287840276480,"bytes_total":289910292480,"degraded_objects":33078,"degraded_total":215898,"degraded_ratio":0.15321123864046912,"misplaced_objects":71,"misplaced_total":215898,"misplaced_ratio":0.00032885899823064594,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":68933,"recovering_keys_per_sec":0,"num_objects_recovered":202,"num_bytes_recovered":827392,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":384},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:54:58.356 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:54:58.356 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:54:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:54:58.989 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:54:59.030 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-16T04:54:58.413001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":441475338,"num_objects":107791,"num_object_clones":0,"num_object_copies":215582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33078,"num_objects_misplaced":71,"num_objects_unfound":0,"num_objects_dirty":107791,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178843,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7594,"num_bytes_recovered":31051795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2028576,"kb_used_data":1022820,"kb_used_omap":12,"kb_used_meta":1005683,"kb_avail":281086944,"statfs":{"total":289910292480,"available":287833030656,"internally_reserved":0,"allocated":1047367680,"data_stored":1046466110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1029819393},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,3,5,14,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19542016,"num_objects":-4771,"num_object_clones":0,"num_object_copies":-9542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":84,"num_objects_misplaced":-286,"num_objects_unfound":0,"num_objects_dirty":-4771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4771,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":827392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002690"},"pg_stats":[{"pgid":"2.f","version":"49'13132","reported_seq":13216,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.951637+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:53.951637+0000","last_peered":"2024-09-16T04:54:53.951637+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:53.951637+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27058176,"num_objects":6606,"num_object_clones":0,"num_object_copies":13212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13132,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3265},{"shards":"0,2","objects":77}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13276","reported_seq":13397,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:53.513357+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:53.513357+0000","last_peered":"2024-09-16T04:54:53.513357+0000","last_clean":"2024-09-16T04:54:53.513357+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:53.513357+0000","last_undegraded":"2024-09-16T04:54:53.513357+0000","last_fullsized":"2024-09-16T04:54:53.513357+0000","mapping_epoch":35,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27516928,"num_objects":6718,"num_object_clones":0,"num_object_copies":13436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10245,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13164","reported_seq":13307,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.430548+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:53.430548+0000","last_peered":"2024-09-16T04:54:53.430548+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:53.430548+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27414552,"num_objects":6694,"num_object_clones":0,"num_object_copies":13388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6694,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9903,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":86},{"shards":"2","objects":3375}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13479","reported_seq":13554,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:53.427353+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:53.427353+0000","last_peered":"2024-09-16T04:54:53.427353+0000","last_clean":"2024-09-16T04:54:53.427353+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:53.427353+0000","last_undegraded":"2024-09-16T04:54:53.427353+0000","last_fullsized":"2024-09-16T04:54:53.427353+0000","mapping_epoch":35,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28119040,"num_objects":6865,"num_object_clones":0,"num_object_copies":13730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13479,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13369","reported_seq":13593,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:53.975083+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:53.975083+0000","last_peered":"2024-09-16T04:54:53.975083+0000","last_clean":"2024-09-16T04:54:53.975083+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:53.975083+0000","last_undegraded":"2024-09-16T04:54:53.975083+0000","last_fullsized":"2024-09-16T04:54:53.975083+0000","mapping_epoch":34,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10401,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13525","reported_seq":13663,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:54.037310+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:54.037310+0000","last_peered":"2024-09-16T04:54:54.037310+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:54.037310+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27803648,"num_objects":6788,"num_object_clones":0,"num_object_copies":13576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10518,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3332},{"shards":"0,2","objects":90}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13264","reported_seq":13415,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:54.067896+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:54.067896+0000","last_peered":"2024-09-16T04:54:54.067896+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:54.067896+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27017216,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9919,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3197},{"shards":"0,2","objects":68}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13246","reported_seq":13466,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:54.037110+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:54.037110+0000","last_peered":"2024-09-16T04:54:54.037110+0000","last_clean":"2024-09-16T04:54:54.037110+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:54.037110+0000","last_undegraded":"2024-09-16T04:54:54.037110+0000","last_fullsized":"2024-09-16T04:54:54.037110+0000","mapping_epoch":34,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13289","reported_seq":13522,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.341676+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:53.341676+0000","last_peered":"2024-09-16T04:54:53.341676+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:53.341676+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27820032,"num_objects":6792,"num_object_clones":0,"num_object_copies":13584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10040,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":89},{"shards":"2","objects":3471}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579490+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579490+0000","last_peered":"2024-09-16T04:54:05.579490+0000","last_clean":"2024-09-16T04:54:05.579490+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579490+0000","last_undegraded":"2024-09-16T04:54:05.579490+0000","last_fullsized":"2024-09-16T04:54:05.579490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13504","reported_seq":13639,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:54.022920+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:54.022920+0000","last_peered":"2024-09-16T04:54:54.022920+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:54.022920+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27500544,"num_objects":6714,"num_object_clones":0,"num_object_copies":13428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10159,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3249},{"shards":"0,2","objects":74}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13225","reported_seq":15295,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.499903+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:53.499903+0000","last_peered":"2024-09-16T04:54:53.499903+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:53.499903+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27665618,"num_objects":6756,"num_object_clones":0,"num_object_copies":13512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2316,"num_objects_misplaced":71,"num_objects_unfound":0,"num_objects_dirty":6756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13197,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":913,"num_bytes_recovered":3723264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":71},{"shards":"2","objects":4369}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13234","reported_seq":20096,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:53.486637+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:53.486637+0000","last_peered":"2024-09-16T04:54:53.486637+0000","last_clean":"2024-09-16T04:54:53.486637+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:53.486637+0000","last_undegraded":"2024-09-16T04:54:53.486637+0000","last_fullsized":"2024-09-16T04:54:53.486637+0000","mapping_epoch":46,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9957,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13266","reported_seq":13368,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.477357+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:53.477357+0000","last_peered":"2024-09-16T04:54:53.477357+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:53.477357+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27075177,"num_objects":6611,"num_object_clones":0,"num_object_copies":13222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13265,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":80},{"shards":"2","objects":3204}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13681","reported_seq":13789,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:53.337958+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:53.337958+0000","last_peered":"2024-09-16T04:54:53.337958+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:53.337958+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28063547,"num_objects":6854,"num_object_clones":0,"num_object_copies":13708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13678,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":103},{"shards":"2","objects":3340}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13454","reported_seq":20253,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:57.744602+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:57.744602+0000","last_peered":"2024-09-16T04:54:57.744602+0000","last_clean":"2024-09-16T04:54:57.744602+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:57.744602+0000","last_undegraded":"2024-09-16T04:54:57.744602+0000","last_fullsized":"2024-09-16T04:54:57.744602+0000","mapping_epoch":48,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10208,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13655","reported_seq":13800,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:54.034175+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:54.034175+0000","last_peered":"2024-09-16T04:54:54.034175+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:54.034175+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27693673,"num_objects":6762,"num_object_clones":0,"num_object_copies":13524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10424,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3238},{"shards":"0,2","objects":83}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":441475319,"num_objects":107789,"num_object_clones":0,"num_object_copies":215578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33078,"num_objects_misplaced":71,"num_objects_unfound":0,"num_objects_dirty":107789,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178841,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7593,"num_bytes_recovered":31051776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1044963328,"data_stored":1044889083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696392,"kb_used_data":353716,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93675448,"statfs":{"total":96636764160,"available":95923658752,"internally_reserved":0,"allocated":362205184,"data_stored":361921858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593793,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638572,"kb_used_data":318104,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93733268,"statfs":{"total":96636764160,"available":95982866432,"internally_reserved":0,"allocated":325738496,"data_stored":325399506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,7,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693612,"kb_used_data":351000,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93678228,"statfs":{"total":96636764160,"available":95926505472,"internally_reserved":0,"allocated":359424000,"data_stored":359144746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":358875136,"data_stored":358871657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325226496,"data_stored":325163281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360861696,"data_stored":360854145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:02.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:02.818 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:02.819 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:02.858 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-16T04:55:02.413969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":432836874,"num_objects":105682,"num_object_clones":0,"num_object_copies":211364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105682,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180952,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7693,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160675,"ondisk_log_size":160675,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2047764,"kb_used_data":1011488,"kb_used_omap":12,"kb_used_meta":1036211,"kb_avail":281067756,"statfs":{"total":289910292480,"available":287813382144,"internally_reserved":0,"allocated":1035763712,"data_stored":1034862142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1061080065},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,4,19,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18599936,"num_objects":-4541,"num_object_clones":0,"num_object_copies":-9082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":-213,"num_objects_unfound":0,"num_objects_dirty":-4541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4541,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":671744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002747"},"pg_stats":[{"pgid":"2.f","version":"49'13273","reported_seq":13352,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:59.069731+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:59.069731+0000","last_peered":"2024-09-16T04:54:59.069731+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:59.069731+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26480640,"num_objects":6465,"num_object_clones":0,"num_object_copies":12930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13273,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3124}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13406","reported_seq":13527,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:58.476479+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:58.476479+0000","last_peered":"2024-09-16T04:54:58.476479+0000","last_clean":"2024-09-16T04:54:58.476479+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:58.476479+0000","last_undegraded":"2024-09-16T04:54:58.476479+0000","last_fullsized":"2024-09-16T04:54:58.476479+0000","mapping_epoch":35,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26984448,"num_objects":6588,"num_object_clones":0,"num_object_copies":13176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10375,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13313","reported_seq":13457,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.443712+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:58.443712+0000","last_peered":"2024-09-16T04:54:58.443712+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:58.443712+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"42'3300","ondisk_log_start":"42'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26804248,"num_objects":6545,"num_object_clones":0,"num_object_copies":13090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10052,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3226}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13618","reported_seq":13693,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:58.404349+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:58.404349+0000","last_peered":"2024-09-16T04:54:58.404349+0000","last_clean":"2024-09-16T04:54:58.404349+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:58.404349+0000","last_undegraded":"2024-09-16T04:54:58.404349+0000","last_fullsized":"2024-09-16T04:54:58.404349+0000","mapping_epoch":35,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27549696,"num_objects":6726,"num_object_clones":0,"num_object_copies":13452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13618,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13510","reported_seq":13734,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:58.983824+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:58.983824+0000","last_peered":"2024-09-16T04:54:58.983824+0000","last_clean":"2024-09-16T04:54:58.983824+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:58.983824+0000","last_undegraded":"2024-09-16T04:54:58.983824+0000","last_fullsized":"2024-09-16T04:54:58.983824+0000","mapping_epoch":34,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27172864,"num_objects":6634,"num_object_clones":0,"num_object_copies":13268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10542,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13675","reported_seq":13813,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.944831+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:58.944831+0000","last_peered":"2024-09-16T04:54:58.944831+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:58.944831+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27189248,"num_objects":6638,"num_object_clones":0,"num_object_copies":13276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10668,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13386","reported_seq":13539,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:59.000743+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:59.000743+0000","last_peered":"2024-09-16T04:54:59.000743+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:59.000743+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26517504,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10041,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3075}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13381","reported_seq":13602,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:58.961328+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:58.961328+0000","last_peered":"2024-09-16T04:54:58.961328+0000","last_clean":"2024-09-16T04:54:58.961328+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:58.961328+0000","last_undegraded":"2024-09-16T04:54:58.961328+0000","last_fullsized":"2024-09-16T04:54:58.961328+0000","mapping_epoch":34,"log_start":"42'3300","ondisk_log_start":"42'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27176960,"num_objects":6635,"num_object_clones":0,"num_object_copies":13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10451,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13417","reported_seq":13648,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.419616+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:58.419616+0000","last_peered":"2024-09-16T04:54:58.419616+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:58.419616+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27295744,"num_objects":6664,"num_object_clones":0,"num_object_copies":13328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10168,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3343}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13654","reported_seq":13789,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:59.055387+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:59.055387+0000","last_peered":"2024-09-16T04:54:59.055387+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:59.055387+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26886144,"num_objects":6564,"num_object_clones":0,"num_object_copies":13128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10309,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3099}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13352","reported_seq":15620,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.531474+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:58.531474+0000","last_peered":"2024-09-16T04:54:58.531474+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:58.531474+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"42'3300","ondisk_log_start":"42'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27145426,"num_objects":6629,"num_object_clones":0,"num_object_copies":13258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13324,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1012,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4341}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13386","reported_seq":20248,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:58.479194+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:58.479194+0000","last_peered":"2024-09-16T04:54:58.479194+0000","last_clean":"2024-09-16T04:54:58.479194+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:58.479194+0000","last_undegraded":"2024-09-16T04:54:58.479194+0000","last_fullsized":"2024-09-16T04:54:58.479194+0000","mapping_epoch":46,"log_start":"42'3300","ondisk_log_start":"42'3300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27000832,"num_objects":6592,"num_object_clones":0,"num_object_copies":13184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10109,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13403","reported_seq":13505,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.472487+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:58.472487+0000","last_peered":"2024-09-16T04:54:58.472487+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:58.472487+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"42'3400","ondisk_log_start":"42'3400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26514025,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13402,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3067}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13842","reported_seq":13950,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.455374+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:58.455374+0000","last_peered":"2024-09-16T04:54:58.455374+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:58.455374+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27404091,"num_objects":6693,"num_object_clones":0,"num_object_copies":13386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13839,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3179}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13454","reported_seq":20253,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:57.744602+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:57.744602+0000","last_peered":"2024-09-16T04:54:57.744602+0000","last_clean":"2024-09-16T04:54:57.744602+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:57.744602+0000","last_undegraded":"2024-09-16T04:54:57.744602+0000","last_fullsized":"2024-09-16T04:54:57.744602+0000","mapping_epoch":48,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10208,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13802","reported_seq":13947,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.965536+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:58.965536+0000","last_peered":"2024-09-16T04:54:58.965536+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:58.965536+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27091561,"num_objects":6615,"num_object_clones":0,"num_object_copies":13230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10571,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3091}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":432836855,"num_objects":105680,"num_object_clones":0,"num_object_copies":211360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105680,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180950,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7692,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1033453568,"data_stored":1033379323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160672,"ondisk_log_size":160672,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699040,"kb_used_data":348048,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93672800,"statfs":{"total":96636764160,"available":95920947200,"internally_reserved":0,"allocated":356401152,"data_stored":356117826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":359394985},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593793,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638572,"kb_used_data":318104,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93733268,"statfs":{"total":96636764160,"available":95982866432,"internally_reserved":0,"allocated":325738496,"data_stored":325399506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,7,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710152,"kb_used_data":345336,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93661688,"statfs":{"total":96636764160,"available":95909568512,"internally_reserved":0,"allocated":353624064,"data_stored":353344810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":353095680,"data_stored":353092201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325226496,"data_stored":325163281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355131392,"data_stored":355123841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:02.859 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:03.635 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:03.637 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:03.733 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-16T04:55:02.413969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":432836874,"num_objects":105682,"num_object_clones":0,"num_object_copies":211364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105682,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180952,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7693,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160675,"ondisk_log_size":160675,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2047764,"kb_used_data":1011488,"kb_used_omap":12,"kb_used_meta":1036211,"kb_avail":281067756,"statfs":{"total":289910292480,"available":287813382144,"internally_reserved":0,"allocated":1035763712,"data_stored":1034862142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1061080065},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,4,19,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18599936,"num_objects":-4541,"num_object_clones":0,"num_object_copies":-9082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":-213,"num_objects_unfound":0,"num_objects_dirty":-4541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4541,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":671744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002747"},"pg_stats":[{"pgid":"2.f","version":"49'13273","reported_seq":13352,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:59.069731+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:54:59.069731+0000","last_peered":"2024-09-16T04:54:59.069731+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:54:59.069731+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"39'3200","ondisk_log_start":"39'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26480640,"num_objects":6465,"num_object_clones":0,"num_object_copies":12930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13273,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3124}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13406","reported_seq":13527,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:58.476479+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:54:58.476479+0000","last_peered":"2024-09-16T04:54:58.476479+0000","last_clean":"2024-09-16T04:54:58.476479+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:54:58.476479+0000","last_undegraded":"2024-09-16T04:54:58.476479+0000","last_fullsized":"2024-09-16T04:54:58.476479+0000","mapping_epoch":35,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26984448,"num_objects":6588,"num_object_clones":0,"num_object_copies":13176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10375,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13313","reported_seq":13457,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.443712+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:54:58.443712+0000","last_peered":"2024-09-16T04:54:58.443712+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:54:58.443712+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"42'3300","ondisk_log_start":"42'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26804248,"num_objects":6545,"num_object_clones":0,"num_object_copies":13090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10052,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3226}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13618","reported_seq":13693,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:58.404349+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:54:58.404349+0000","last_peered":"2024-09-16T04:54:58.404349+0000","last_clean":"2024-09-16T04:54:58.404349+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:54:58.404349+0000","last_undegraded":"2024-09-16T04:54:58.404349+0000","last_fullsized":"2024-09-16T04:54:58.404349+0000","mapping_epoch":35,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27549696,"num_objects":6726,"num_object_clones":0,"num_object_copies":13452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13618,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13510","reported_seq":13734,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:58.983824+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:54:58.983824+0000","last_peered":"2024-09-16T04:54:58.983824+0000","last_clean":"2024-09-16T04:54:58.983824+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:54:58.983824+0000","last_undegraded":"2024-09-16T04:54:58.983824+0000","last_fullsized":"2024-09-16T04:54:58.983824+0000","mapping_epoch":34,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27172864,"num_objects":6634,"num_object_clones":0,"num_object_copies":13268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10542,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13675","reported_seq":13813,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.944831+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:54:58.944831+0000","last_peered":"2024-09-16T04:54:58.944831+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:54:58.944831+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27189248,"num_objects":6638,"num_object_clones":0,"num_object_copies":13276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10668,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13386","reported_seq":13539,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:59.000743+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:54:59.000743+0000","last_peered":"2024-09-16T04:54:59.000743+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:54:59.000743+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"39'3300","ondisk_log_start":"39'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26517504,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10041,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3075}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13381","reported_seq":13602,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:58.961328+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:54:58.961328+0000","last_peered":"2024-09-16T04:54:58.961328+0000","last_clean":"2024-09-16T04:54:58.961328+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:54:58.961328+0000","last_undegraded":"2024-09-16T04:54:58.961328+0000","last_fullsized":"2024-09-16T04:54:58.961328+0000","mapping_epoch":34,"log_start":"42'3300","ondisk_log_start":"42'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27176960,"num_objects":6635,"num_object_clones":0,"num_object_copies":13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10451,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13417","reported_seq":13648,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.419616+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:54:58.419616+0000","last_peered":"2024-09-16T04:54:58.419616+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:54:58.419616+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27295744,"num_objects":6664,"num_object_clones":0,"num_object_copies":13328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10168,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3343}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13654","reported_seq":13789,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:59.055387+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:54:59.055387+0000","last_peered":"2024-09-16T04:54:59.055387+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:54:59.055387+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26886144,"num_objects":6564,"num_object_clones":0,"num_object_copies":13128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10309,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3099}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13352","reported_seq":15620,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.531474+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:54:58.531474+0000","last_peered":"2024-09-16T04:54:58.531474+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:54:58.531474+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"42'3300","ondisk_log_start":"42'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27145426,"num_objects":6629,"num_object_clones":0,"num_object_copies":13258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13324,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1012,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4341}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13386","reported_seq":20248,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:58.479194+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:54:58.479194+0000","last_peered":"2024-09-16T04:54:58.479194+0000","last_clean":"2024-09-16T04:54:58.479194+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:54:58.479194+0000","last_undegraded":"2024-09-16T04:54:58.479194+0000","last_fullsized":"2024-09-16T04:54:58.479194+0000","mapping_epoch":46,"log_start":"42'3300","ondisk_log_start":"42'3300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27000832,"num_objects":6592,"num_object_clones":0,"num_object_copies":13184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10109,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13403","reported_seq":13505,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.472487+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:54:58.472487+0000","last_peered":"2024-09-16T04:54:58.472487+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:54:58.472487+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"42'3400","ondisk_log_start":"42'3400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26514025,"num_objects":6474,"num_object_clones":0,"num_object_copies":12948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13402,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3067}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13842","reported_seq":13950,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.455374+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:54:58.455374+0000","last_peered":"2024-09-16T04:54:58.455374+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:54:58.455374+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27404091,"num_objects":6693,"num_object_clones":0,"num_object_copies":13386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13839,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3179}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582828+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582828+0000","last_peered":"2024-09-16T04:54:05.582828+0000","last_clean":"2024-09-16T04:54:05.582828+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582828+0000","last_undegraded":"2024-09-16T04:54:05.582828+0000","last_fullsized":"2024-09-16T04:54:05.582828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13454","reported_seq":20253,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:57.744602+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:54:57.744602+0000","last_peered":"2024-09-16T04:54:57.744602+0000","last_clean":"2024-09-16T04:54:57.744602+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:54:57.744602+0000","last_undegraded":"2024-09-16T04:54:57.744602+0000","last_fullsized":"2024-09-16T04:54:57.744602+0000","mapping_epoch":48,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27623424,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10208,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13802","reported_seq":13947,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:54:58.965536+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:54:58.965536+0000","last_peered":"2024-09-16T04:54:58.965536+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:54:58.965536+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27091561,"num_objects":6615,"num_object_clones":0,"num_object_copies":13230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10571,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3091}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":432836855,"num_objects":105680,"num_object_clones":0,"num_object_copies":211360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105680,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":180950,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7692,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1033453568,"data_stored":1033379323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160672,"ondisk_log_size":160672,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699040,"kb_used_data":348048,"kb_used_omap":4,"kb_used_meta":350971,"kb_avail":93672800,"statfs":{"total":96636764160,"available":95920947200,"internally_reserved":0,"allocated":356401152,"data_stored":356117826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":359394985},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,8,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593793,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638572,"kb_used_data":318104,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93733268,"statfs":{"total":96636764160,"available":95982866432,"internally_reserved":0,"allocated":325738496,"data_stored":325399506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,7,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710152,"kb_used_data":345336,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93661688,"statfs":{"total":96636764160,"available":95909568512,"internally_reserved":0,"allocated":353624064,"data_stored":353344810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":353095680,"data_stored":353092201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325226496,"data_stored":325163281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355131392,"data_stored":355123841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:03.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:55:03.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'13273", 'reported_seq': 13352, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:59.069731+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:54:59.069731+0000', 'last_peered': '2024-09-16T04:54:59.069731+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:54:59.069731+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "39'3200", 'ondisk_log_start': "39'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26480640, 'num_objects': 6465, 'num_object_clones': 0, 'num_object_copies': 12930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6465, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13273, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3124}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:03.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:55:03.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'13313", 'reported_seq': 13457, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:58.443712+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:54:58.443712+0000', 'last_peered': '2024-09-16T04:54:58.443712+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:54:58.443712+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "42'3300", 'ondisk_log_start': "42'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26804248, 'num_objects': 6545, 'num_object_clones': 0, 'num_object_copies': 13090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6545, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10052, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:03.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:55:03.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'13675", 'reported_seq': 13813, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:58.944831+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:54:58.944831+0000', 'last_peered': '2024-09-16T04:54:58.944831+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:54:58.944831+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'3600", 'ondisk_log_start': "45'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27189248, 'num_objects': 6638, 'num_object_clones': 0, 'num_object_copies': 13276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6638, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10668, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3182}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:03.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:55:03.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'13386", 'reported_seq': 13539, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:59.000743+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:54:59.000743+0000', 'last_peered': '2024-09-16T04:54:59.000743+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:54:59.000743+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "39'3300", 'ondisk_log_start': "39'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26517504, 'num_objects': 6474, 'num_object_clones': 0, 'num_object_copies': 12948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10041, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3075}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:03.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:55:03.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'13417", 'reported_seq': 13648, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:58.419616+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:54:58.419616+0000', 'last_peered': '2024-09-16T04:54:58.419616+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:54:58.419616+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'3400", 'ondisk_log_start': "45'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27295744, 'num_objects': 6664, 'num_object_clones': 0, 'num_object_copies': 13328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6664, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10168, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3343}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:03.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:55:03.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'13654", 'reported_seq': 13789, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:59.055387+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:54:59.055387+0000', 'last_peered': '2024-09-16T04:54:59.055387+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:54:59.055387+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'3600", 'ondisk_log_start': "45'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26886144, 'num_objects': 6564, 'num_object_clones': 0, 'num_object_copies': 13128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6564, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10309, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3099}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:03.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:55:03.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'13352", 'reported_seq': 15620, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:58.531474+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:54:58.531474+0000', 'last_peered': '2024-09-16T04:54:58.531474+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:54:58.531474+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "42'3300", 'ondisk_log_start': "42'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27145426, 'num_objects': 6629, 'num_object_clones': 0, 'num_object_copies': 13258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2288, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6629, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13324, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1012, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4341}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:03.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:55:03.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'13403", 'reported_seq': 13505, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:58.472487+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:54:58.472487+0000', 'last_peered': '2024-09-16T04:54:58.472487+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:54:58.472487+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "42'3400", 'ondisk_log_start': "42'3400", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26514025, 'num_objects': 6474, 'num_object_clones': 0, 'num_object_copies': 12948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13402, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3067}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:03.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:55:03.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'13842", 'reported_seq': 13950, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:58.455374+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:54:58.455374+0000', 'last_peered': '2024-09-16T04:54:58.455374+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:54:58.455374+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'3800", 'ondisk_log_start': "45'3800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27404091, 'num_objects': 6693, 'num_object_clones': 0, 'num_object_copies': 13386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6693, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13839, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3179}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:03.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:55:03.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'13802", 'reported_seq': 13947, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:54:58.965536+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:54:58.965536+0000', 'last_peered': '2024-09-16T04:54:58.965536+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:54:58.965536+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'3800", 'ondisk_log_start': "45'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27091561, 'num_objects': 6615, 'num_object_clones': 0, 'num_object_copies': 13230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6615, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10571, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3091}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:03.737 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:55:04.728 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:04.728 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32958/206616 objects degraded (15.951%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":452,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":103308,"data_bytes":423112970,"bytes_used":2078973952,"bytes_avail":287831318528,"bytes_total":289910292480,"degraded_objects":32958,"degraded_total":206616,"degraded_ratio":0.15951330003484726,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":55965,"recovering_keys_per_sec":0,"num_objects_recovered":292,"num_bytes_recovered":671744,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":576},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:55:04.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:55:04.804 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:05.588 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:05.591 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:05.642 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-16T04:55:04.414394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":423112970,"num_objects":103308,"num_object_clones":0,"num_object_copies":206616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183326,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7785,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2030248,"kb_used_data":992552,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281085272,"statfs":{"total":289910292480,"available":287831318528,"internally_reserved":0,"allocated":1016373248,"data_stored":1015471678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1062521857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,3,9,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28323840,"num_objects":-6915,"num_object_clones":0,"num_object_copies":-13830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-79,"num_objects_misplaced":-213,"num_objects_unfound":0,"num_objects_dirty":-6915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6915,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":671744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002749"},"pg_stats":[{"pgid":"2.f","version":"49'13424","reported_seq":13508,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:04.057808+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:04.057808+0000","last_peered":"2024-09-16T04:55:04.057808+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:04.057808+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25862144,"num_objects":6314,"num_object_clones":0,"num_object_copies":12628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13424,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13584","reported_seq":13705,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:03.531064+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:03.531064+0000","last_peered":"2024-09-16T04:55:03.531064+0000","last_clean":"2024-09-16T04:55:03.531064+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:03.531064+0000","last_undegraded":"2024-09-16T04:55:03.531064+0000","last_fullsized":"2024-09-16T04:55:03.531064+0000","mapping_epoch":35,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26255360,"num_objects":6410,"num_object_clones":0,"num_object_copies":12820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10553,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13454","reported_seq":13598,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.556235+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:03.556235+0000","last_peered":"2024-09-16T04:55:03.556235+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:03.556235+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26226712,"num_objects":6404,"num_object_clones":0,"num_object_copies":12808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6404,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10193,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3085}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13767","reported_seq":13842,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:03.447786+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:03.447786+0000","last_peered":"2024-09-16T04:55:03.447786+0000","last_clean":"2024-09-16T04:55:03.447786+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:03.447786+0000","last_undegraded":"2024-09-16T04:55:03.447786+0000","last_fullsized":"2024-09-16T04:55:03.447786+0000","mapping_epoch":35,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26939392,"num_objects":6577,"num_object_clones":0,"num_object_copies":13154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13767,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13656","reported_seq":13880,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:04.065748+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:04.065748+0000","last_peered":"2024-09-16T04:55:04.065748+0000","last_clean":"2024-09-16T04:55:04.065748+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:04.065748+0000","last_undegraded":"2024-09-16T04:55:04.065748+0000","last_fullsized":"2024-09-16T04:55:04.065748+0000","mapping_epoch":34,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26574848,"num_objects":6488,"num_object_clones":0,"num_object_copies":12976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10688,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13806","reported_seq":13945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.986105+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:03.986105+0000","last_peered":"2024-09-16T04:55:03.986105+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:03.986105+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26652672,"num_objects":6507,"num_object_clones":0,"num_object_copies":13014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10799,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13530","reported_seq":13682,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:04.010734+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:04.010734+0000","last_peered":"2024-09-16T04:55:04.010734+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:04.010734+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25927680,"num_objects":6330,"num_object_clones":0,"num_object_copies":12660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10185,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13543","reported_seq":13763,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:04.099511+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:04.099511+0000","last_peered":"2024-09-16T04:55:04.099511+0000","last_clean":"2024-09-16T04:55:04.099511+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:04.099511+0000","last_undegraded":"2024-09-16T04:55:04.099511+0000","last_fullsized":"2024-09-16T04:55:04.099511+0000","mapping_epoch":34,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26513408,"num_objects":6473,"num_object_clones":0,"num_object_copies":12946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10613,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13557","reported_seq":13785,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.491786+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:03.491786+0000","last_peered":"2024-09-16T04:55:03.491786+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:03.491786+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722304,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10308,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3203}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13789","reported_seq":13924,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:04.089042+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:04.089042+0000","last_peered":"2024-09-16T04:55:04.089042+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:04.089042+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10444,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2964}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13506","reported_seq":15958,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.557050+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:03.557050+0000","last_peered":"2024-09-16T04:55:03.557050+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:03.557050+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26514642,"num_objects":6475,"num_object_clones":0,"num_object_copies":12950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13478,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1104,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4279}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13547","reported_seq":20409,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:03.485892+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:03.485892+0000","last_peered":"2024-09-16T04:55:03.485892+0000","last_clean":"2024-09-16T04:55:03.485892+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:03.485892+0000","last_undegraded":"2024-09-16T04:55:03.485892+0000","last_fullsized":"2024-09-16T04:55:03.485892+0000","mapping_epoch":46,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26341376,"num_objects":6431,"num_object_clones":0,"num_object_copies":12862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10270,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13548","reported_seq":13651,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.545142+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:03.545142+0000","last_peered":"2024-09-16T04:55:03.545142+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:03.545142+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25920105,"num_objects":6329,"num_object_clones":0,"num_object_copies":12658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13547,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13993","reported_seq":14101,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.531330+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:03.531330+0000","last_peered":"2024-09-16T04:55:03.531330+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:03.531330+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26785595,"num_objects":6542,"num_object_clones":0,"num_object_copies":13084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13990,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3028}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13585","reported_seq":20384,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:02.796630+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:02.796630+0000","last_peered":"2024-09-16T04:55:02.796630+0000","last_clean":"2024-09-16T04:55:02.796630+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:02.796630+0000","last_undegraded":"2024-09-16T04:55:02.796630+0000","last_fullsized":"2024-09-16T04:55:02.796630+0000","mapping_epoch":48,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27086848,"num_objects":6613,"num_object_clones":0,"num_object_copies":13226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10339,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13957","reported_seq":14101,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:04.116685+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:04.116685+0000","last_peered":"2024-09-16T04:55:04.116685+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:04.116685+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26456681,"num_objects":6460,"num_object_clones":0,"num_object_copies":12920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10726,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2936}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":423112951,"num_objects":103306,"num_object_clones":0,"num_object_copies":206612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103306,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183324,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7784,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1014075392,"data_stored":1014001147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160846,"ondisk_log_size":160846,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698516,"kb_used_data":341952,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93673324,"statfs":{"total":96636764160,"available":95921483776,"internally_reserved":0,"allocated":350158848,"data_stored":349875522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593794,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635780,"kb_used_data":311156,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93736060,"statfs":{"total":96636764160,"available":95985725440,"internally_reserved":0,"allocated":318623744,"data_stored":318284754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695952,"kb_used_data":339444,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93675888,"statfs":{"total":96636764160,"available":95924109312,"internally_reserved":0,"allocated":347590656,"data_stored":347311402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347049984,"data_stored":347046505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318070784,"data_stored":318007569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":348954624,"data_stored":348947073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:08.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:09.562 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:09.564 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:09.631 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-16T04:55:08.415213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":422392074,"num_objects":103132,"num_object_clones":0,"num_object_copies":206264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183502,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7785,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160825,"ondisk_log_size":160825,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2015020,"kb_used_data":985640,"kb_used_omap":12,"kb_used_meta":1029299,"kb_avail":281100500,"statfs":{"total":289910292480,"available":287846912000,"internally_reserved":0,"allocated":1009295360,"data_stored":1008393790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1054002177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,6,3,6,10,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19730432,"num_objects":-4817,"num_object_clones":0,"num_object_copies":-9634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-120,"num_objects_misplaced":-71,"num_objects_unfound":0,"num_objects_dirty":-4817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4817,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":258048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002666"},"pg_stats":[{"pgid":"2.f","version":"49'13424","reported_seq":13508,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:04.057808+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:04.057808+0000","last_peered":"2024-09-16T04:55:04.057808+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:04.057808+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25862144,"num_objects":6314,"num_object_clones":0,"num_object_copies":12628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13424,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13584","reported_seq":13705,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:03.531064+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:03.531064+0000","last_peered":"2024-09-16T04:55:03.531064+0000","last_clean":"2024-09-16T04:55:03.531064+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:03.531064+0000","last_undegraded":"2024-09-16T04:55:03.531064+0000","last_fullsized":"2024-09-16T04:55:03.531064+0000","mapping_epoch":35,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26255360,"num_objects":6410,"num_object_clones":0,"num_object_copies":12820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10553,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13454","reported_seq":13598,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.556235+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:03.556235+0000","last_peered":"2024-09-16T04:55:03.556235+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:03.556235+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26226712,"num_objects":6404,"num_object_clones":0,"num_object_copies":12808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6404,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10193,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3085}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13767","reported_seq":13842,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:03.447786+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:03.447786+0000","last_peered":"2024-09-16T04:55:03.447786+0000","last_clean":"2024-09-16T04:55:03.447786+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:03.447786+0000","last_undegraded":"2024-09-16T04:55:03.447786+0000","last_fullsized":"2024-09-16T04:55:03.447786+0000","mapping_epoch":35,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26939392,"num_objects":6577,"num_object_clones":0,"num_object_copies":13154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13767,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13656","reported_seq":13880,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:04.065748+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:04.065748+0000","last_peered":"2024-09-16T04:55:04.065748+0000","last_clean":"2024-09-16T04:55:04.065748+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:04.065748+0000","last_undegraded":"2024-09-16T04:55:04.065748+0000","last_fullsized":"2024-09-16T04:55:04.065748+0000","mapping_epoch":34,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26574848,"num_objects":6488,"num_object_clones":0,"num_object_copies":12976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10688,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13806","reported_seq":13945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.986105+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:03.986105+0000","last_peered":"2024-09-16T04:55:03.986105+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:03.986105+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26652672,"num_objects":6507,"num_object_clones":0,"num_object_copies":13014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10799,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13530","reported_seq":13682,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:04.010734+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:04.010734+0000","last_peered":"2024-09-16T04:55:04.010734+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:04.010734+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25927680,"num_objects":6330,"num_object_clones":0,"num_object_copies":12660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10185,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13543","reported_seq":13763,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:04.099511+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:04.099511+0000","last_peered":"2024-09-16T04:55:04.099511+0000","last_clean":"2024-09-16T04:55:04.099511+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:04.099511+0000","last_undegraded":"2024-09-16T04:55:04.099511+0000","last_fullsized":"2024-09-16T04:55:04.099511+0000","mapping_epoch":34,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26513408,"num_objects":6473,"num_object_clones":0,"num_object_copies":12946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10613,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13557","reported_seq":13785,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.491786+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:03.491786+0000","last_peered":"2024-09-16T04:55:03.491786+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:03.491786+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722304,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10308,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3203}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13789","reported_seq":13924,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:04.089042+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:04.089042+0000","last_peered":"2024-09-16T04:55:04.089042+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:04.089042+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10444,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2964}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13506","reported_seq":15958,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.557050+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:03.557050+0000","last_peered":"2024-09-16T04:55:03.557050+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:03.557050+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26514642,"num_objects":6475,"num_object_clones":0,"num_object_copies":12950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13478,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1104,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4279}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13547","reported_seq":20409,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:03.485892+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:03.485892+0000","last_peered":"2024-09-16T04:55:03.485892+0000","last_clean":"2024-09-16T04:55:03.485892+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:03.485892+0000","last_undegraded":"2024-09-16T04:55:03.485892+0000","last_fullsized":"2024-09-16T04:55:03.485892+0000","mapping_epoch":46,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26341376,"num_objects":6431,"num_object_clones":0,"num_object_copies":12862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10270,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13548","reported_seq":13651,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.545142+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:03.545142+0000","last_peered":"2024-09-16T04:55:03.545142+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:03.545142+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25920105,"num_objects":6329,"num_object_clones":0,"num_object_copies":12658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13547,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13993","reported_seq":14101,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.531330+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:03.531330+0000","last_peered":"2024-09-16T04:55:03.531330+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:03.531330+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26785595,"num_objects":6542,"num_object_clones":0,"num_object_copies":13084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13990,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3028}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13761","reported_seq":20560,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:07.883973+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:07.883973+0000","last_peered":"2024-09-16T04:55:07.883973+0000","last_clean":"2024-09-16T04:55:07.883973+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:07.883973+0000","last_undegraded":"2024-09-16T04:55:07.883973+0000","last_fullsized":"2024-09-16T04:55:07.883973+0000","mapping_epoch":48,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10515,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13957","reported_seq":14101,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:04.116685+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:04.116685+0000","last_peered":"2024-09-16T04:55:04.116685+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:04.116685+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26456681,"num_objects":6460,"num_object_clones":0,"num_object_copies":12920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10726,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2936}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":422392055,"num_objects":103130,"num_object_clones":0,"num_object_copies":206260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103130,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183500,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7784,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1006833664,"data_stored":1006759419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698516,"kb_used_data":341952,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93673324,"statfs":{"total":96636764160,"available":95921483776,"internally_reserved":0,"allocated":350158848,"data_stored":349875522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593795,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620552,"kb_used_data":304244,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93751288,"statfs":{"total":96636764160,"available":96001318912,"internally_reserved":0,"allocated":311545856,"data_stored":311206866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695952,"kb_used_data":339444,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93675888,"statfs":{"total":96636764160,"available":95924109312,"internally_reserved":0,"allocated":347590656,"data_stored":347311402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347049984,"data_stored":347046505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310829056,"data_stored":310765841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":348954624,"data_stored":348947073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:09.632 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:10.412 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:10.414 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:10.504 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-16T04:55:08.415213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":422392074,"num_objects":103132,"num_object_clones":0,"num_object_copies":206264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183502,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7785,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160825,"ondisk_log_size":160825,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2015020,"kb_used_data":985640,"kb_used_omap":12,"kb_used_meta":1029299,"kb_avail":281100500,"statfs":{"total":289910292480,"available":287846912000,"internally_reserved":0,"allocated":1009295360,"data_stored":1008393790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1054002177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,6,3,6,10,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19730432,"num_objects":-4817,"num_object_clones":0,"num_object_copies":-9634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-120,"num_objects_misplaced":-71,"num_objects_unfound":0,"num_objects_dirty":-4817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4817,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":258048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002666"},"pg_stats":[{"pgid":"2.f","version":"49'13424","reported_seq":13508,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:04.057808+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:04.057808+0000","last_peered":"2024-09-16T04:55:04.057808+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:04.057808+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25862144,"num_objects":6314,"num_object_clones":0,"num_object_copies":12628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13424,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2973}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13584","reported_seq":13705,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:03.531064+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:03.531064+0000","last_peered":"2024-09-16T04:55:03.531064+0000","last_clean":"2024-09-16T04:55:03.531064+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:03.531064+0000","last_undegraded":"2024-09-16T04:55:03.531064+0000","last_fullsized":"2024-09-16T04:55:03.531064+0000","mapping_epoch":35,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26255360,"num_objects":6410,"num_object_clones":0,"num_object_copies":12820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10553,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13454","reported_seq":13598,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.556235+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:03.556235+0000","last_peered":"2024-09-16T04:55:03.556235+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:03.556235+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'3400","ondisk_log_start":"45'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26226712,"num_objects":6404,"num_object_clones":0,"num_object_copies":12808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6404,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10193,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3085}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13767","reported_seq":13842,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:03.447786+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:03.447786+0000","last_peered":"2024-09-16T04:55:03.447786+0000","last_clean":"2024-09-16T04:55:03.447786+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:03.447786+0000","last_undegraded":"2024-09-16T04:55:03.447786+0000","last_fullsized":"2024-09-16T04:55:03.447786+0000","mapping_epoch":35,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26939392,"num_objects":6577,"num_object_clones":0,"num_object_copies":13154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13767,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13656","reported_seq":13880,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:04.065748+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:04.065748+0000","last_peered":"2024-09-16T04:55:04.065748+0000","last_clean":"2024-09-16T04:55:04.065748+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:04.065748+0000","last_undegraded":"2024-09-16T04:55:04.065748+0000","last_fullsized":"2024-09-16T04:55:04.065748+0000","mapping_epoch":34,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26574848,"num_objects":6488,"num_object_clones":0,"num_object_copies":12976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10688,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13806","reported_seq":13945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.986105+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:03.986105+0000","last_peered":"2024-09-16T04:55:03.986105+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:03.986105+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26652672,"num_objects":6507,"num_object_clones":0,"num_object_copies":13014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10799,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13530","reported_seq":13682,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:04.010734+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:04.010734+0000","last_peered":"2024-09-16T04:55:04.010734+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:04.010734+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25927680,"num_objects":6330,"num_object_clones":0,"num_object_copies":12660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10185,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2931}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13543","reported_seq":13763,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:04.099511+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:04.099511+0000","last_peered":"2024-09-16T04:55:04.099511+0000","last_clean":"2024-09-16T04:55:04.099511+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:04.099511+0000","last_undegraded":"2024-09-16T04:55:04.099511+0000","last_fullsized":"2024-09-16T04:55:04.099511+0000","mapping_epoch":34,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26513408,"num_objects":6473,"num_object_clones":0,"num_object_copies":12946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10613,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13557","reported_seq":13785,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.491786+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:03.491786+0000","last_peered":"2024-09-16T04:55:03.491786+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:03.491786+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722304,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10308,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3203}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13789","reported_seq":13924,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:04.089042+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:04.089042+0000","last_peered":"2024-09-16T04:55:04.089042+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:04.089042+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26333184,"num_objects":6429,"num_object_clones":0,"num_object_copies":12858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10444,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2964}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13506","reported_seq":15958,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.557050+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:03.557050+0000","last_peered":"2024-09-16T04:55:03.557050+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:03.557050+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26514642,"num_objects":6475,"num_object_clones":0,"num_object_copies":12950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13478,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1104,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4279}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13547","reported_seq":20409,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:03.485892+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:03.485892+0000","last_peered":"2024-09-16T04:55:03.485892+0000","last_clean":"2024-09-16T04:55:03.485892+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:03.485892+0000","last_undegraded":"2024-09-16T04:55:03.485892+0000","last_fullsized":"2024-09-16T04:55:03.485892+0000","mapping_epoch":46,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26341376,"num_objects":6431,"num_object_clones":0,"num_object_copies":12862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10270,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13548","reported_seq":13651,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.545142+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:03.545142+0000","last_peered":"2024-09-16T04:55:03.545142+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:03.545142+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25920105,"num_objects":6329,"num_object_clones":0,"num_object_copies":12658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13547,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'13993","reported_seq":14101,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:03.531330+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:03.531330+0000","last_peered":"2024-09-16T04:55:03.531330+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:03.531330+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26785595,"num_objects":6542,"num_object_clones":0,"num_object_copies":13084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13990,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3028}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13761","reported_seq":20560,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:07.883973+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:07.883973+0000","last_peered":"2024-09-16T04:55:07.883973+0000","last_clean":"2024-09-16T04:55:07.883973+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:07.883973+0000","last_undegraded":"2024-09-16T04:55:07.883973+0000","last_fullsized":"2024-09-16T04:55:07.883973+0000","mapping_epoch":48,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10515,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'13957","reported_seq":14101,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:04.116685+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:04.116685+0000","last_peered":"2024-09-16T04:55:04.116685+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:04.116685+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26456681,"num_objects":6460,"num_object_clones":0,"num_object_copies":12920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10726,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2936}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":422392055,"num_objects":103130,"num_object_clones":0,"num_object_copies":206260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103130,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":183500,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7784,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1006833664,"data_stored":1006759419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698516,"kb_used_data":341952,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93673324,"statfs":{"total":96636764160,"available":95921483776,"internally_reserved":0,"allocated":350158848,"data_stored":349875522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593795,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620552,"kb_used_data":304244,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93751288,"statfs":{"total":96636764160,"available":96001318912,"internally_reserved":0,"allocated":311545856,"data_stored":311206866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695952,"kb_used_data":339444,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93675888,"statfs":{"total":96636764160,"available":95924109312,"internally_reserved":0,"allocated":347590656,"data_stored":347311402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347049984,"data_stored":347046505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310829056,"data_stored":310765841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":348954624,"data_stored":348947073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:10.506 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:55:10.506 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'13424", 'reported_seq': 13508, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:04.057808+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:55:04.057808+0000', 'last_peered': '2024-09-16T04:55:04.057808+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:55:04.057808+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'3400", 'ondisk_log_start': "45'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25862144, 'num_objects': 6314, 'num_object_clones': 0, 'num_object_copies': 12628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6314, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13424, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2973}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:10.506 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:55:10.506 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'13454", 'reported_seq': 13598, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:03.556235+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:55:03.556235+0000', 'last_peered': '2024-09-16T04:55:03.556235+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:55:03.556235+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'3400", 'ondisk_log_start': "45'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26226712, 'num_objects': 6404, 'num_object_clones': 0, 'num_object_copies': 12808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6404, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10193, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3085}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:10.506 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:55:10.506 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'13806", 'reported_seq': 13945, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:03.986105+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:55:03.986105+0000', 'last_peered': '2024-09-16T04:55:03.986105+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:55:03.986105+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'3800", 'ondisk_log_start': "45'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26652672, 'num_objects': 6507, 'num_object_clones': 0, 'num_object_copies': 13014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6507, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10799, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3051}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:10.506 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:55:10.506 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'13530", 'reported_seq': 13682, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:04.010734+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:55:04.010734+0000', 'last_peered': '2024-09-16T04:55:04.010734+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:55:04.010734+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'3500", 'ondisk_log_start': "45'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25927680, 'num_objects': 6330, 'num_object_clones': 0, 'num_object_copies': 12660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6330, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10185, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2931}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:10.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:55:10.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'13557", 'reported_seq': 13785, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:03.491786+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:55:03.491786+0000', 'last_peered': '2024-09-16T04:55:03.491786+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:55:03.491786+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'3500", 'ondisk_log_start': "45'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26722304, 'num_objects': 6524, 'num_object_clones': 0, 'num_object_copies': 13048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6524, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10308, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:10.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:55:10.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'13789", 'reported_seq': 13924, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:04.089042+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:55:04.089042+0000', 'last_peered': '2024-09-16T04:55:04.089042+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:55:04.089042+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'3700", 'ondisk_log_start': "45'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26333184, 'num_objects': 6429, 'num_object_clones': 0, 'num_object_copies': 12858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10444, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2964}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:10.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:55:10.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'13506", 'reported_seq': 15958, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:03.557050+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:55:03.557050+0000', 'last_peered': '2024-09-16T04:55:03.557050+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:55:03.557050+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'3500", 'ondisk_log_start': "45'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26514642, 'num_objects': 6475, 'num_object_clones': 0, 'num_object_copies': 12950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2196, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6475, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13478, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1104, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4279}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:10.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:55:10.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'13548", 'reported_seq': 13651, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:03.545142+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:55:03.545142+0000', 'last_peered': '2024-09-16T04:55:03.545142+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:55:03.545142+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'3500", 'ondisk_log_start': "45'3500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25920105, 'num_objects': 6329, 'num_object_clones': 0, 'num_object_copies': 12658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6329, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13547, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2922}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:10.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:55:10.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'13993", 'reported_seq': 14101, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:03.531330+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:55:03.531330+0000', 'last_peered': '2024-09-16T04:55:03.531330+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:55:03.531330+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'3900", 'ondisk_log_start': "45'3900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26785595, 'num_objects': 6542, 'num_object_clones': 0, 'num_object_copies': 13084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6542, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13990, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3028}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:10.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:55:10.508 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'13957", 'reported_seq': 14101, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:04.116685+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:55:04.116685+0000', 'last_peered': '2024-09-16T04:55:04.116685+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:55:04.116685+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'3900", 'ondisk_log_start': "45'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26456681, 'num_objects': 6460, 'num_object_clones': 0, 'num_object_copies': 12920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6460, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10726, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2936}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:10.508 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:55:11.535 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:11.536 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32875/201930 objects degraded (16.280%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":459,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":100965,"data_bytes":413516042,"bytes_used":2060050432,"bytes_avail":287850242048,"bytes_total":289910292480,"degraded_objects":32875,"degraded_total":201930,"degraded_ratio":0.16280394196008519,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":21499,"recovering_keys_per_sec":0,"num_objects_recovered":274,"num_bytes_recovered":258048,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":568},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:55:11.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:55:11.621 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:12.531 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:12.532 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:12.603 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-16T04:55:12.420788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":413516042,"num_objects":100965,"num_object_clones":0,"num_object_copies":201930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100965,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185669,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7868,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161092,"ondisk_log_size":161092,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2011768,"kb_used_data":974064,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281103752,"statfs":{"total":289910292480,"available":287850242048,"internally_reserved":0,"allocated":997441536,"data_stored":996539966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1062521857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,6,3,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19320832,"num_objects":-4717,"num_object_clones":0,"num_object_copies":-9434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4717,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007316"},"pg_stats":[{"pgid":"2.f","version":"49'13591","reported_seq":13671,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:08.985500+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:08.985500+0000","last_peered":"2024-09-16T04:55:08.985500+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:08.985500+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25178112,"num_objects":6147,"num_object_clones":0,"num_object_copies":12294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13591,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2806}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13741","reported_seq":13862,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:08.586763+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:08.586763+0000","last_peered":"2024-09-16T04:55:08.586763+0000","last_clean":"2024-09-16T04:55:08.586763+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:08.586763+0000","last_undegraded":"2024-09-16T04:55:08.586763+0000","last_fullsized":"2024-09-16T04:55:08.586763+0000","mapping_epoch":35,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25612288,"num_objects":6253,"num_object_clones":0,"num_object_copies":12506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10710,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13591","reported_seq":13735,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:08.554393+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:08.554393+0000","last_peered":"2024-09-16T04:55:08.554393+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:08.554393+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'3500","ondisk_log_start":"45'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25665560,"num_objects":6267,"num_object_clones":0,"num_object_copies":12534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6267,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10330,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2948}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'13911","reported_seq":13986,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:08.579510+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:08.579510+0000","last_peered":"2024-09-16T04:55:08.579510+0000","last_clean":"2024-09-16T04:55:08.579510+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:08.579510+0000","last_undegraded":"2024-09-16T04:55:08.579510+0000","last_fullsized":"2024-09-16T04:55:08.579510+0000","mapping_epoch":35,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26349568,"num_objects":6433,"num_object_clones":0,"num_object_copies":12866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13911,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13799","reported_seq":14023,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:08.954395+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:08.954395+0000","last_peered":"2024-09-16T04:55:08.954395+0000","last_clean":"2024-09-16T04:55:08.954395+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:08.954395+0000","last_undegraded":"2024-09-16T04:55:08.954395+0000","last_fullsized":"2024-09-16T04:55:08.954395+0000","mapping_epoch":34,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25989120,"num_objects":6345,"num_object_clones":0,"num_object_copies":12690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10831,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'13952","reported_seq":14091,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:09.097971+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:09.097971+0000","last_peered":"2024-09-16T04:55:09.097971+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:09.097971+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26054656,"num_objects":6361,"num_object_clones":0,"num_object_copies":12722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10945,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2905}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13671","reported_seq":13825,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:09.052497+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:09.052497+0000","last_peered":"2024-09-16T04:55:09.052497+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:09.052497+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25350144,"num_objects":6189,"num_object_clones":0,"num_object_copies":12378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10326,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2790}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13682","reported_seq":13903,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:09.117729+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:09.117729+0000","last_peered":"2024-09-16T04:55:09.117729+0000","last_clean":"2024-09-16T04:55:09.117729+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:09.117729+0000","last_undegraded":"2024-09-16T04:55:09.117729+0000","last_fullsized":"2024-09-16T04:55:09.117729+0000","mapping_epoch":34,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25944064,"num_objects":6334,"num_object_clones":0,"num_object_copies":12668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10752,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13700","reported_seq":13933,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:08.453045+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:08.453045+0000","last_peered":"2024-09-16T04:55:08.453045+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:08.453045+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26136576,"num_objects":6381,"num_object_clones":0,"num_object_copies":12762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10451,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3060}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'13937","reported_seq":14072,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:09.047213+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:09.047213+0000","last_peered":"2024-09-16T04:55:09.047213+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:09.047213+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25726976,"num_objects":6281,"num_object_clones":0,"num_object_copies":12562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10592,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2816}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13678","reported_seq":16294,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:08.595853+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:08.595853+0000","last_peered":"2024-09-16T04:55:08.595853+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:08.595853+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25810130,"num_objects":6303,"num_object_clones":0,"num_object_copies":12606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2113,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13650,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1187,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4190}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13677","reported_seq":20539,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:08.520784+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:08.520784+0000","last_peered":"2024-09-16T04:55:08.520784+0000","last_clean":"2024-09-16T04:55:08.520784+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:08.520784+0000","last_undegraded":"2024-09-16T04:55:08.520784+0000","last_fullsized":"2024-09-16T04:55:08.520784+0000","mapping_epoch":46,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25808896,"num_objects":6301,"num_object_clones":0,"num_object_copies":12602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10400,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13687","reported_seq":13790,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:08.564879+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:08.564879+0000","last_peered":"2024-09-16T04:55:08.564879+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:08.564879+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'3600","ondisk_log_start":"45'3600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25350761,"num_objects":6190,"num_object_clones":0,"num_object_copies":12380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13686,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2783}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14115","reported_seq":14223,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:08.529121+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:08.529121+0000","last_peered":"2024-09-16T04:55:08.529121+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:08.529121+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26285883,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14112,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2906}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13761","reported_seq":20560,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:07.883973+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:07.883973+0000","last_peered":"2024-09-16T04:55:07.883973+0000","last_clean":"2024-09-16T04:55:07.883973+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:07.883973+0000","last_undegraded":"2024-09-16T04:55:07.883973+0000","last_fullsized":"2024-09-16T04:55:07.883973+0000","mapping_epoch":48,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10515,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14096","reported_seq":14241,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:09.081788+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:09.081788+0000","last_peered":"2024-09-16T04:55:09.081788+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:09.081788+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25887337,"num_objects":6321,"num_object_clones":0,"num_object_copies":12642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10865,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2797}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":413516023,"num_objects":100963,"num_object_clones":0,"num_object_copies":201926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185667,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7867,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":994750464,"data_stored":994676219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161089,"ondisk_log_size":161089,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697064,"kb_used_data":336324,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93674776,"statfs":{"total":96636764160,"available":95922970624,"internally_reserved":0,"allocated":344395776,"data_stored":344112450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,2,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593795,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620552,"kb_used_data":304244,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93751288,"statfs":{"total":96636764160,"available":96001318912,"internally_reserved":0,"allocated":311545856,"data_stored":311206866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":694152,"kb_used_data":333496,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93677688,"statfs":{"total":96636764160,"available":95925952512,"internally_reserved":0,"allocated":341499904,"data_stored":341220650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":340918272,"data_stored":340914793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310829056,"data_stored":310765841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":343003136,"data_stored":342995585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:15.605 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:16.459 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:16.461 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:16.524 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-09-16T04:55:16.421880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":404058378,"num_objects":98656,"num_object_clones":0,"num_object_copies":197312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98656,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187978,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7970,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160701,"ondisk_log_size":160701,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2005900,"kb_used_data":955732,"kb_used_omap":12,"kb_used_meta":1050099,"kb_avail":281109620,"statfs":{"total":289910292480,"available":287856250880,"internally_reserved":0,"allocated":978669568,"data_stored":977767998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1075301377},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,2,7,9,8,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19054592,"num_objects":-4652,"num_object_clones":0,"num_object_copies":-9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4652,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007486"},"pg_stats":[{"pgid":"2.f","version":"49'13708","reported_seq":13793,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.984192+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:13.984192+0000","last_peered":"2024-09-16T04:55:13.984192+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:13.984192+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24698880,"num_objects":6030,"num_object_clones":0,"num_object_copies":12060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13708,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2689}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13885","reported_seq":14006,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:13.579559+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:13.579559+0000","last_peered":"2024-09-16T04:55:13.579559+0000","last_clean":"2024-09-16T04:55:13.579559+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:13.579559+0000","last_undegraded":"2024-09-16T04:55:13.579559+0000","last_fullsized":"2024-09-16T04:55:13.579559+0000","mapping_epoch":35,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25022464,"num_objects":6109,"num_object_clones":0,"num_object_copies":12218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10854,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13741","reported_seq":13885,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.548658+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:13.548658+0000","last_peered":"2024-09-16T04:55:13.548658+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:13.548658+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051160,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10480,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2798}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14037","reported_seq":14112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:13.562606+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:13.562606+0000","last_peered":"2024-09-16T04:55:13.562606+0000","last_clean":"2024-09-16T04:55:13.562606+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:13.562606+0000","last_undegraded":"2024-09-16T04:55:13.562606+0000","last_fullsized":"2024-09-16T04:55:13.562606+0000","mapping_epoch":35,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25833472,"num_objects":6307,"num_object_clones":0,"num_object_copies":12614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14037,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13946","reported_seq":14170,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:14.135437+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:14.135437+0000","last_peered":"2024-09-16T04:55:14.135437+0000","last_clean":"2024-09-16T04:55:14.135437+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:14.135437+0000","last_undegraded":"2024-09-16T04:55:14.135437+0000","last_fullsized":"2024-09-16T04:55:14.135437+0000","mapping_epoch":34,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25387008,"num_objects":6198,"num_object_clones":0,"num_object_copies":12396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10978,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14091","reported_seq":14230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:14.083349+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:14.083349+0000","last_peered":"2024-09-16T04:55:14.083349+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:14.083349+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25485312,"num_objects":6222,"num_object_clones":0,"num_object_copies":12444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11084,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2766}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13847","reported_seq":13995,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:14.119342+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:14.119342+0000","last_peered":"2024-09-16T04:55:14.119342+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:14.119342+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24629248,"num_objects":6013,"num_object_clones":0,"num_object_copies":12026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10502,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13828","reported_seq":14049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:14.043709+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:14.043709+0000","last_peered":"2024-09-16T04:55:14.043709+0000","last_clean":"2024-09-16T04:55:14.043709+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:14.043709+0000","last_undegraded":"2024-09-16T04:55:14.043709+0000","last_fullsized":"2024-09-16T04:55:14.043709+0000","mapping_epoch":34,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10898,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13857","reported_seq":14087,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.592712+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:13.592712+0000","last_peered":"2024-09-16T04:55:13.592712+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:13.592712+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10608,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2903}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14071","reported_seq":14206,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:14.078984+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:14.078984+0000","last_peered":"2024-09-16T04:55:14.078984+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:14.078984+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25178112,"num_objects":6147,"num_object_clones":0,"num_object_copies":12294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10726,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2682}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13817","reported_seq":16639,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.623109+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:13.623109+0000","last_peered":"2024-09-16T04:55:13.623109+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:13.623109+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25240786,"num_objects":6164,"num_object_clones":0,"num_object_copies":12328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2011,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13789,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1289,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4153}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13811","reported_seq":20673,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:13.560645+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:13.560645+0000","last_peered":"2024-09-16T04:55:13.560645+0000","last_clean":"2024-09-16T04:55:13.560645+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:13.560645+0000","last_undegraded":"2024-09-16T04:55:13.560645+0000","last_fullsized":"2024-09-16T04:55:13.560645+0000","mapping_epoch":46,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25260032,"num_objects":6167,"num_object_clones":0,"num_object_copies":12334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10534,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13851","reported_seq":13954,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.623053+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:13.623053+0000","last_peered":"2024-09-16T04:55:13.623053+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:13.623053+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24679017,"num_objects":6026,"num_object_clones":0,"num_object_copies":12052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13850,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2619}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14259","reported_seq":14367,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.434100+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:13.434100+0000","last_peered":"2024-09-16T04:55:13.434100+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:13.434100+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25696059,"num_objects":6276,"num_object_clones":0,"num_object_copies":12552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2762}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13904","reported_seq":20703,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:12.932159+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:12.932159+0000","last_peered":"2024-09-16T04:55:12.932159+0000","last_clean":"2024-09-16T04:55:12.932159+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:12.932159+0000","last_undegraded":"2024-09-16T04:55:12.932159+0000","last_fullsized":"2024-09-16T04:55:12.932159+0000","mapping_epoch":48,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25780224,"num_objects":6294,"num_object_clones":0,"num_object_copies":12588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10658,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14245","reported_seq":14390,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:14.010203+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:14.010203+0000","last_peered":"2024-09-16T04:55:14.010203+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:14.010203+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25277033,"num_objects":6172,"num_object_clones":0,"num_object_copies":12344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11014,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":404058359,"num_objects":98654,"num_object_clones":0,"num_object_copies":197308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98654,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187976,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7969,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":975941632,"data_stored":975867387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160698,"ondisk_log_size":160698,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695448,"kb_used_data":330560,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93676392,"statfs":{"total":96636764160,"available":95924625408,"internally_reserved":0,"allocated":338493440,"data_stored":338210114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,2,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593796,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617788,"kb_used_data":297324,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93754052,"statfs":{"total":96636764160,"available":96004149248,"internally_reserved":0,"allocated":304459776,"data_stored":304120786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,7,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692664,"kb_used_data":327848,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93679176,"statfs":{"total":96636764160,"available":95927476224,"internally_reserved":0,"allocated":335716352,"data_stored":335437098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,1,1,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335118336,"data_stored":335114857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303689728,"data_stored":303626513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":337133568,"data_stored":337126017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:16.525 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:17.396 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:17.398 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:17.508 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-09-16T04:55:16.421880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":404058378,"num_objects":98656,"num_object_clones":0,"num_object_copies":197312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98656,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187978,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7970,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160701,"ondisk_log_size":160701,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2005900,"kb_used_data":955732,"kb_used_omap":12,"kb_used_meta":1050099,"kb_avail":281109620,"statfs":{"total":289910292480,"available":287856250880,"internally_reserved":0,"allocated":978669568,"data_stored":977767998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1075301377},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,2,7,9,8,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19054592,"num_objects":-4652,"num_object_clones":0,"num_object_copies":-9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4652,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007486"},"pg_stats":[{"pgid":"2.f","version":"49'13708","reported_seq":13793,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.984192+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:13.984192+0000","last_peered":"2024-09-16T04:55:13.984192+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:13.984192+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24698880,"num_objects":6030,"num_object_clones":0,"num_object_copies":12060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13708,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2689}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13885","reported_seq":14006,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:13.579559+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:13.579559+0000","last_peered":"2024-09-16T04:55:13.579559+0000","last_clean":"2024-09-16T04:55:13.579559+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:13.579559+0000","last_undegraded":"2024-09-16T04:55:13.579559+0000","last_fullsized":"2024-09-16T04:55:13.579559+0000","mapping_epoch":35,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25022464,"num_objects":6109,"num_object_clones":0,"num_object_copies":12218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10854,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13741","reported_seq":13885,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.548658+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:13.548658+0000","last_peered":"2024-09-16T04:55:13.548658+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:13.548658+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051160,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10480,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2798}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14037","reported_seq":14112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:13.562606+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:13.562606+0000","last_peered":"2024-09-16T04:55:13.562606+0000","last_clean":"2024-09-16T04:55:13.562606+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:13.562606+0000","last_undegraded":"2024-09-16T04:55:13.562606+0000","last_fullsized":"2024-09-16T04:55:13.562606+0000","mapping_epoch":35,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25833472,"num_objects":6307,"num_object_clones":0,"num_object_copies":12614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14037,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13946","reported_seq":14170,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:14.135437+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:14.135437+0000","last_peered":"2024-09-16T04:55:14.135437+0000","last_clean":"2024-09-16T04:55:14.135437+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:14.135437+0000","last_undegraded":"2024-09-16T04:55:14.135437+0000","last_fullsized":"2024-09-16T04:55:14.135437+0000","mapping_epoch":34,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25387008,"num_objects":6198,"num_object_clones":0,"num_object_copies":12396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10978,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14091","reported_seq":14230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:14.083349+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:14.083349+0000","last_peered":"2024-09-16T04:55:14.083349+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:14.083349+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25485312,"num_objects":6222,"num_object_clones":0,"num_object_copies":12444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11084,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2766}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13847","reported_seq":13995,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:14.119342+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:14.119342+0000","last_peered":"2024-09-16T04:55:14.119342+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:14.119342+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24629248,"num_objects":6013,"num_object_clones":0,"num_object_copies":12026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10502,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13828","reported_seq":14049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:14.043709+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:14.043709+0000","last_peered":"2024-09-16T04:55:14.043709+0000","last_clean":"2024-09-16T04:55:14.043709+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:14.043709+0000","last_undegraded":"2024-09-16T04:55:14.043709+0000","last_fullsized":"2024-09-16T04:55:14.043709+0000","mapping_epoch":34,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10898,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13857","reported_seq":14087,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.592712+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:13.592712+0000","last_peered":"2024-09-16T04:55:13.592712+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:13.592712+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10608,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2903}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14071","reported_seq":14206,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:14.078984+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:14.078984+0000","last_peered":"2024-09-16T04:55:14.078984+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:14.078984+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25178112,"num_objects":6147,"num_object_clones":0,"num_object_copies":12294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10726,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2682}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13817","reported_seq":16639,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.623109+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:13.623109+0000","last_peered":"2024-09-16T04:55:13.623109+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:13.623109+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25240786,"num_objects":6164,"num_object_clones":0,"num_object_copies":12328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2011,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13789,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1289,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4153}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13811","reported_seq":20673,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:13.560645+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:13.560645+0000","last_peered":"2024-09-16T04:55:13.560645+0000","last_clean":"2024-09-16T04:55:13.560645+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:13.560645+0000","last_undegraded":"2024-09-16T04:55:13.560645+0000","last_fullsized":"2024-09-16T04:55:13.560645+0000","mapping_epoch":46,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25260032,"num_objects":6167,"num_object_clones":0,"num_object_copies":12334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10534,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13851","reported_seq":13954,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.623053+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:13.623053+0000","last_peered":"2024-09-16T04:55:13.623053+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:13.623053+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24679017,"num_objects":6026,"num_object_clones":0,"num_object_copies":12052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13850,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2619}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14259","reported_seq":14367,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.434100+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:13.434100+0000","last_peered":"2024-09-16T04:55:13.434100+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:13.434100+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25696059,"num_objects":6276,"num_object_clones":0,"num_object_copies":12552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2762}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'13904","reported_seq":20703,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:12.932159+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:12.932159+0000","last_peered":"2024-09-16T04:55:12.932159+0000","last_clean":"2024-09-16T04:55:12.932159+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:12.932159+0000","last_undegraded":"2024-09-16T04:55:12.932159+0000","last_fullsized":"2024-09-16T04:55:12.932159+0000","mapping_epoch":48,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25780224,"num_objects":6294,"num_object_clones":0,"num_object_copies":12588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10658,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14245","reported_seq":14390,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:14.010203+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:14.010203+0000","last_peered":"2024-09-16T04:55:14.010203+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:14.010203+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25277033,"num_objects":6172,"num_object_clones":0,"num_object_copies":12344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11014,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":404058359,"num_objects":98654,"num_object_clones":0,"num_object_copies":197308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98654,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187976,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7969,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":975941632,"data_stored":975867387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160698,"ondisk_log_size":160698,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695448,"kb_used_data":330560,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93676392,"statfs":{"total":96636764160,"available":95924625408,"internally_reserved":0,"allocated":338493440,"data_stored":338210114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,2,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593796,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617788,"kb_used_data":297324,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93754052,"statfs":{"total":96636764160,"available":96004149248,"internally_reserved":0,"allocated":304459776,"data_stored":304120786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,7,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692664,"kb_used_data":327848,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93679176,"statfs":{"total":96636764160,"available":95927476224,"internally_reserved":0,"allocated":335716352,"data_stored":335437098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,1,1,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335118336,"data_stored":335114857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303689728,"data_stored":303626513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":337133568,"data_stored":337126017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:17.510 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:55:17.511 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'13708", 'reported_seq': 13793, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:13.984192+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:55:13.984192+0000', 'last_peered': '2024-09-16T04:55:13.984192+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:55:13.984192+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'3700", 'ondisk_log_start': "45'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24698880, 'num_objects': 6030, 'num_object_clones': 0, 'num_object_copies': 12060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6030, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13708, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2689}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:17.511 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:55:17.511 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'13741", 'reported_seq': 13885, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:13.548658+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:55:13.548658+0000', 'last_peered': '2024-09-16T04:55:13.548658+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:55:13.548658+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'3700", 'ondisk_log_start': "45'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25051160, 'num_objects': 6117, 'num_object_clones': 0, 'num_object_copies': 12234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6117, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10480, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2798}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:17.511 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:55:17.511 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'14091", 'reported_seq': 14230, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:14.083349+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:55:14.083349+0000', 'last_peered': '2024-09-16T04:55:14.083349+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:55:14.083349+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'4000", 'ondisk_log_start': "45'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25485312, 'num_objects': 6222, 'num_object_clones': 0, 'num_object_copies': 12444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6222, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11084, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2766}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:17.512 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:55:17.512 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'13847", 'reported_seq': 13995, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:14.119342+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:55:14.119342+0000', 'last_peered': '2024-09-16T04:55:14.119342+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:55:14.119342+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'3800", 'ondisk_log_start': "45'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24629248, 'num_objects': 6013, 'num_object_clones': 0, 'num_object_copies': 12026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6013, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10502, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:17.512 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:55:17.512 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'13857", 'reported_seq': 14087, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:13.592712+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:55:13.592712+0000', 'last_peered': '2024-09-16T04:55:13.592712+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:55:13.592712+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'3800", 'ondisk_log_start': "45'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25493504, 'num_objects': 6224, 'num_object_clones': 0, 'num_object_copies': 12448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10608, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2903}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:17.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:55:17.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'14071", 'reported_seq': 14206, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:14.078984+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:55:14.078984+0000', 'last_peered': '2024-09-16T04:55:14.078984+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:55:14.078984+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'4000", 'ondisk_log_start': "45'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25178112, 'num_objects': 6147, 'num_object_clones': 0, 'num_object_copies': 12294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6147, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10726, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2682}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:17.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:55:17.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'13817", 'reported_seq': 16639, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:13.623109+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:55:13.623109+0000', 'last_peered': '2024-09-16T04:55:13.623109+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:55:13.623109+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'3800", 'ondisk_log_start': "45'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25240786, 'num_objects': 6164, 'num_object_clones': 0, 'num_object_copies': 12328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2011, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6164, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13789, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1289, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4153}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:17.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:55:17.513 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'13851", 'reported_seq': 13954, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:13.623053+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:55:13.623053+0000', 'last_peered': '2024-09-16T04:55:13.623053+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:55:13.623053+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'3800", 'ondisk_log_start': "45'3800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24679017, 'num_objects': 6026, 'num_object_clones': 0, 'num_object_copies': 12052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13850, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2619}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:17.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:55:17.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'14259", 'reported_seq': 14367, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:13.434100+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:55:13.434100+0000', 'last_peered': '2024-09-16T04:55:13.434100+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:55:13.434100+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'4200", 'ondisk_log_start': "45'4200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25696059, 'num_objects': 6276, 'num_object_clones': 0, 'num_object_copies': 12552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6276, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14256, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2762}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:17.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:55:17.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'14245", 'reported_seq': 14390, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:14.010203+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:55:14.010203+0000', 'last_peered': '2024-09-16T04:55:14.010203+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:55:14.010203+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'4200", 'ondisk_log_start': "45'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25277033, 'num_objects': 6172, 'num_object_clones': 0, 'num_object_copies': 12344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6172, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11014, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:17.515 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:55:18.581 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:18.581 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32773/197008 objects degraded (16.635%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":466,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":98504,"data_bytes":403435786,"bytes_used":2055372800,"bytes_avail":287854919680,"bytes_total":289910292480,"degraded_objects":32773,"degraded_total":197008,"degraded_ratio":0.16635365061317306,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":185,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":400},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:55:18.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:55:18.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:19.461 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:19.463 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:19.512 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-16T04:55:18.422311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":403435786,"num_objects":98504,"num_object_clones":0,"num_object_copies":197008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188130,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7970,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2007200,"kb_used_data":948708,"kb_used_omap":12,"kb_used_meta":1058419,"kb_avail":281108320,"statfs":{"total":289910292480,"available":287854919680,"internally_reserved":0,"allocated":971476992,"data_stored":970575422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1083821057},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,5,10,7,2,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19677184,"num_objects":-4804,"num_object_clones":0,"num_object_copies":-9608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4804,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007525"},"pg_stats":[{"pgid":"2.f","version":"49'13708","reported_seq":13793,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.984192+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:13.984192+0000","last_peered":"2024-09-16T04:55:13.984192+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:13.984192+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24698880,"num_objects":6030,"num_object_clones":0,"num_object_copies":12060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13708,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2689}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'13885","reported_seq":14006,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:13.579559+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:13.579559+0000","last_peered":"2024-09-16T04:55:13.579559+0000","last_clean":"2024-09-16T04:55:13.579559+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:13.579559+0000","last_undegraded":"2024-09-16T04:55:13.579559+0000","last_fullsized":"2024-09-16T04:55:13.579559+0000","mapping_epoch":35,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25022464,"num_objects":6109,"num_object_clones":0,"num_object_copies":12218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10854,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13741","reported_seq":13885,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.548658+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:13.548658+0000","last_peered":"2024-09-16T04:55:13.548658+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:13.548658+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'3700","ondisk_log_start":"45'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051160,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10480,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2798}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14037","reported_seq":14112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:13.562606+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:13.562606+0000","last_peered":"2024-09-16T04:55:13.562606+0000","last_clean":"2024-09-16T04:55:13.562606+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:13.562606+0000","last_undegraded":"2024-09-16T04:55:13.562606+0000","last_fullsized":"2024-09-16T04:55:13.562606+0000","mapping_epoch":35,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25833472,"num_objects":6307,"num_object_clones":0,"num_object_copies":12614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14037,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'13946","reported_seq":14170,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:14.135437+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:14.135437+0000","last_peered":"2024-09-16T04:55:14.135437+0000","last_clean":"2024-09-16T04:55:14.135437+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:14.135437+0000","last_undegraded":"2024-09-16T04:55:14.135437+0000","last_fullsized":"2024-09-16T04:55:14.135437+0000","mapping_epoch":34,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25387008,"num_objects":6198,"num_object_clones":0,"num_object_copies":12396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10978,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14091","reported_seq":14230,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:14.083349+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:14.083349+0000","last_peered":"2024-09-16T04:55:14.083349+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:14.083349+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25485312,"num_objects":6222,"num_object_clones":0,"num_object_copies":12444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11084,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2766}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13847","reported_seq":13995,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:14.119342+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:14.119342+0000","last_peered":"2024-09-16T04:55:14.119342+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:14.119342+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24629248,"num_objects":6013,"num_object_clones":0,"num_object_copies":12026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10502,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13828","reported_seq":14049,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:14.043709+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:14.043709+0000","last_peered":"2024-09-16T04:55:14.043709+0000","last_clean":"2024-09-16T04:55:14.043709+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:14.043709+0000","last_undegraded":"2024-09-16T04:55:14.043709+0000","last_fullsized":"2024-09-16T04:55:14.043709+0000","mapping_epoch":34,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10898,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'13857","reported_seq":14087,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.592712+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:13.592712+0000","last_peered":"2024-09-16T04:55:13.592712+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:13.592712+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25493504,"num_objects":6224,"num_object_clones":0,"num_object_copies":12448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10608,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2903}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14071","reported_seq":14206,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:14.078984+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:14.078984+0000","last_peered":"2024-09-16T04:55:14.078984+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:14.078984+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25178112,"num_objects":6147,"num_object_clones":0,"num_object_copies":12294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10726,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2682}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13817","reported_seq":16639,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.623109+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:13.623109+0000","last_peered":"2024-09-16T04:55:13.623109+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:13.623109+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25240786,"num_objects":6164,"num_object_clones":0,"num_object_copies":12328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2011,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13789,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1289,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4153}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13811","reported_seq":20673,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:13.560645+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:13.560645+0000","last_peered":"2024-09-16T04:55:13.560645+0000","last_clean":"2024-09-16T04:55:13.560645+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:13.560645+0000","last_undegraded":"2024-09-16T04:55:13.560645+0000","last_fullsized":"2024-09-16T04:55:13.560645+0000","mapping_epoch":46,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25260032,"num_objects":6167,"num_object_clones":0,"num_object_copies":12334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10534,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13851","reported_seq":13954,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.623053+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:13.623053+0000","last_peered":"2024-09-16T04:55:13.623053+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:13.623053+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24679017,"num_objects":6026,"num_object_clones":0,"num_object_copies":12052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13850,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2619}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14259","reported_seq":14367,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:13.434100+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:13.434100+0000","last_peered":"2024-09-16T04:55:13.434100+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:13.434100+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25696059,"num_objects":6276,"num_object_clones":0,"num_object_copies":12552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2762}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14056","reported_seq":20855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:18.045047+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:18.045047+0000","last_peered":"2024-09-16T04:55:18.045047+0000","last_clean":"2024-09-16T04:55:18.045047+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:18.045047+0000","last_undegraded":"2024-09-16T04:55:18.045047+0000","last_fullsized":"2024-09-16T04:55:18.045047+0000","mapping_epoch":48,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25157632,"num_objects":6142,"num_object_clones":0,"num_object_copies":12284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10810,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14245","reported_seq":14390,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:14.010203+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:14.010203+0000","last_peered":"2024-09-16T04:55:14.010203+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:14.010203+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25277033,"num_objects":6172,"num_object_clones":0,"num_object_copies":12344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11014,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":403435767,"num_objects":98502,"num_object_clones":0,"num_object_copies":197004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188128,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7969,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":968523776,"data_stored":968449531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695448,"kb_used_data":330560,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93676392,"statfs":{"total":96636764160,"available":95924625408,"internally_reserved":0,"allocated":338493440,"data_stored":338210114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,2,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593797,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619088,"kb_used_data":290300,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93752752,"statfs":{"total":96636764160,"available":96002818048,"internally_reserved":0,"allocated":297267200,"data_stored":296928210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692664,"kb_used_data":327848,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93679176,"statfs":{"total":96636764160,"available":95927476224,"internally_reserved":0,"allocated":335716352,"data_stored":335437098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,1,1,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335118336,"data_stored":335114857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296271872,"data_stored":296208657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":337133568,"data_stored":337126017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:22.515 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:23.256 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:23.258 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:23.309 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-16T04:55:22.423272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":394371338,"num_objects":96291,"num_object_clones":0,"num_object_copies":192582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32682,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96291,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190343,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8061,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1974552,"kb_used_data":936860,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281140968,"statfs":{"total":289910292480,"available":287888351232,"internally_reserved":0,"allocated":959344640,"data_stored":958443070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1062521857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,8,8,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19144704,"num_objects":-4674,"num_object_clones":0,"num_object_copies":-9348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4674,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007600"},"pg_stats":[{"pgid":"2.f","version":"49'13838","reported_seq":13923,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:19.060043+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:19.060043+0000","last_peered":"2024-09-16T04:55:19.060043+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:19.060043+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24166400,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13838,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2559}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14019","reported_seq":14138,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:18.623884+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:18.623884+0000","last_peered":"2024-09-16T04:55:18.623884+0000","last_clean":"2024-09-16T04:55:18.623884+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:18.623884+0000","last_undegraded":"2024-09-16T04:55:18.623884+0000","last_fullsized":"2024-09-16T04:55:18.623884+0000","mapping_epoch":35,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24473600,"num_objects":5975,"num_object_clones":0,"num_object_copies":11950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10988,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13868","reported_seq":14011,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:18.650893+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:18.650893+0000","last_peered":"2024-09-16T04:55:18.650893+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:18.650893+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24530968,"num_objects":5990,"num_object_clones":0,"num_object_copies":11980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10607,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2671}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14193","reported_seq":14268,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:18.604902+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:18.604902+0000","last_peered":"2024-09-16T04:55:18.604902+0000","last_clean":"2024-09-16T04:55:18.604902+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:18.604902+0000","last_undegraded":"2024-09-16T04:55:18.604902+0000","last_fullsized":"2024-09-16T04:55:18.604902+0000","mapping_epoch":35,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25194496,"num_objects":6151,"num_object_clones":0,"num_object_copies":12302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14193,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'14108","reported_seq":14332,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:19.152696+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:19.152696+0000","last_peered":"2024-09-16T04:55:19.152696+0000","last_clean":"2024-09-16T04:55:19.152696+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:19.152696+0000","last_undegraded":"2024-09-16T04:55:19.152696+0000","last_fullsized":"2024-09-16T04:55:19.152696+0000","mapping_epoch":34,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24723456,"num_objects":6036,"num_object_clones":0,"num_object_copies":12072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11140,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14242","reported_seq":14381,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:19.130170+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:19.130170+0000","last_peered":"2024-09-16T04:55:19.130170+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:19.130170+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24866816,"num_objects":6071,"num_object_clones":0,"num_object_copies":12142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11235,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13990","reported_seq":14144,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:19.111907+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:19.111907+0000","last_peered":"2024-09-16T04:55:19.111907+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:19.111907+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24043520,"num_objects":5870,"num_object_clones":0,"num_object_copies":11740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10645,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2471}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13975","reported_seq":14196,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:19.154030+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:19.154030+0000","last_peered":"2024-09-16T04:55:19.154030+0000","last_clean":"2024-09-16T04:55:19.154030+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:19.154030+0000","last_undegraded":"2024-09-16T04:55:19.154030+0000","last_fullsized":"2024-09-16T04:55:19.154030+0000","mapping_epoch":34,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24743936,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11045,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14015","reported_seq":14248,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:18.573783+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:18.573783+0000","last_peered":"2024-09-16T04:55:18.573783+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:18.573783+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24846336,"num_objects":6066,"num_object_clones":0,"num_object_copies":12132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10766,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2745}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14214","reported_seq":14349,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:19.129070+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:19.129070+0000","last_peered":"2024-09-16T04:55:19.129070+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:19.129070+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24592384,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10869,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2539}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13965","reported_seq":16969,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:18.649907+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:18.649907+0000","last_peered":"2024-09-16T04:55:18.649907+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:18.649907+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24634578,"num_objects":6016,"num_object_clones":0,"num_object_copies":12032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13937,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1380,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4096}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13989","reported_seq":20851,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:18.580014+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:18.580014+0000","last_peered":"2024-09-16T04:55:18.580014+0000","last_clean":"2024-09-16T04:55:18.580014+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:18.580014+0000","last_undegraded":"2024-09-16T04:55:18.580014+0000","last_fullsized":"2024-09-16T04:55:18.580014+0000","mapping_epoch":46,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24530944,"num_objects":5989,"num_object_clones":0,"num_object_copies":11978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10712,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13989","reported_seq":14092,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:18.623539+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:18.623539+0000","last_peered":"2024-09-16T04:55:18.623539+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:18.623539+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113769,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13988,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2481}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14409","reported_seq":14517,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:18.540976+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:18.540976+0000","last_peered":"2024-09-16T04:55:18.540976+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:18.540976+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25081659,"num_objects":6126,"num_object_clones":0,"num_object_copies":12252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14406,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2612}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14056","reported_seq":20855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:18.045047+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:18.045047+0000","last_peered":"2024-09-16T04:55:18.045047+0000","last_clean":"2024-09-16T04:55:18.045047+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:18.045047+0000","last_undegraded":"2024-09-16T04:55:18.045047+0000","last_fullsized":"2024-09-16T04:55:18.045047+0000","mapping_epoch":48,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25157632,"num_objects":6142,"num_object_clones":0,"num_object_copies":12284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10810,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14393","reported_seq":14538,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:19.176336+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:19.176336+0000","last_peered":"2024-09-16T04:55:19.176336+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:19.176336+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24670825,"num_objects":6024,"num_object_clones":0,"num_object_copies":12048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11162,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2500}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":394371319,"num_objects":96289,"num_object_clones":0,"num_object_copies":192578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32682,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190341,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8060,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":956375040,"data_stored":956300795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681164,"kb_used_data":324604,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93690676,"statfs":{"total":96636764160,"available":95939252224,"internally_reserved":0,"allocated":332394496,"data_stored":332111170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593797,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619088,"kb_used_data":290300,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93752752,"statfs":{"total":96636764160,"available":96002818048,"internally_reserved":0,"allocated":297267200,"data_stored":296928210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674300,"kb_used_data":321956,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93697540,"statfs":{"total":96636764160,"available":95946280960,"internally_reserved":0,"allocated":329682944,"data_stored":329403690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":329097216,"data_stored":329093737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296271872,"data_stored":296208657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331005952,"data_stored":330998401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:23.310 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:24.159 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:24.160 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:24.276 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-16T04:55:22.423272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":394371338,"num_objects":96291,"num_object_clones":0,"num_object_copies":192582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32682,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96291,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190343,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8061,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1974552,"kb_used_data":936860,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281140968,"statfs":{"total":289910292480,"available":287888351232,"internally_reserved":0,"allocated":959344640,"data_stored":958443070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1062521857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,8,8,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19144704,"num_objects":-4674,"num_object_clones":0,"num_object_copies":-9348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4674,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007600"},"pg_stats":[{"pgid":"2.f","version":"49'13838","reported_seq":13923,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:19.060043+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:19.060043+0000","last_peered":"2024-09-16T04:55:19.060043+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:19.060043+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24166400,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13838,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2559}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14019","reported_seq":14138,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:18.623884+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:18.623884+0000","last_peered":"2024-09-16T04:55:18.623884+0000","last_clean":"2024-09-16T04:55:18.623884+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:18.623884+0000","last_undegraded":"2024-09-16T04:55:18.623884+0000","last_fullsized":"2024-09-16T04:55:18.623884+0000","mapping_epoch":35,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24473600,"num_objects":5975,"num_object_clones":0,"num_object_copies":11950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10988,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'13868","reported_seq":14011,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:18.650893+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:18.650893+0000","last_peered":"2024-09-16T04:55:18.650893+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:18.650893+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'3800","ondisk_log_start":"45'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24530968,"num_objects":5990,"num_object_clones":0,"num_object_copies":11980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10607,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2671}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14193","reported_seq":14268,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:18.604902+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:18.604902+0000","last_peered":"2024-09-16T04:55:18.604902+0000","last_clean":"2024-09-16T04:55:18.604902+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:18.604902+0000","last_undegraded":"2024-09-16T04:55:18.604902+0000","last_fullsized":"2024-09-16T04:55:18.604902+0000","mapping_epoch":35,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25194496,"num_objects":6151,"num_object_clones":0,"num_object_copies":12302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14193,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'14108","reported_seq":14332,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:19.152696+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:19.152696+0000","last_peered":"2024-09-16T04:55:19.152696+0000","last_clean":"2024-09-16T04:55:19.152696+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:19.152696+0000","last_undegraded":"2024-09-16T04:55:19.152696+0000","last_fullsized":"2024-09-16T04:55:19.152696+0000","mapping_epoch":34,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24723456,"num_objects":6036,"num_object_clones":0,"num_object_copies":12072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11140,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14242","reported_seq":14381,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:19.130170+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:19.130170+0000","last_peered":"2024-09-16T04:55:19.130170+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:19.130170+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24866816,"num_objects":6071,"num_object_clones":0,"num_object_copies":12142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11235,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'13990","reported_seq":14144,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:19.111907+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:19.111907+0000","last_peered":"2024-09-16T04:55:19.111907+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:19.111907+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24043520,"num_objects":5870,"num_object_clones":0,"num_object_copies":11740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10645,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2471}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'13975","reported_seq":14196,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:19.154030+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:19.154030+0000","last_peered":"2024-09-16T04:55:19.154030+0000","last_clean":"2024-09-16T04:55:19.154030+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:19.154030+0000","last_undegraded":"2024-09-16T04:55:19.154030+0000","last_fullsized":"2024-09-16T04:55:19.154030+0000","mapping_epoch":34,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24743936,"num_objects":6041,"num_object_clones":0,"num_object_copies":12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11045,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14015","reported_seq":14248,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:18.573783+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:18.573783+0000","last_peered":"2024-09-16T04:55:18.573783+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:18.573783+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24846336,"num_objects":6066,"num_object_clones":0,"num_object_copies":12132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10766,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2745}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14214","reported_seq":14349,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:19.129070+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:19.129070+0000","last_peered":"2024-09-16T04:55:19.129070+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:19.129070+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24592384,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10869,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2539}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'13965","reported_seq":16969,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:18.649907+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:18.649907+0000","last_peered":"2024-09-16T04:55:18.649907+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:18.649907+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24634578,"num_objects":6016,"num_object_clones":0,"num_object_copies":12032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13937,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1380,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4096}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'13989","reported_seq":20851,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:18.580014+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:18.580014+0000","last_peered":"2024-09-16T04:55:18.580014+0000","last_clean":"2024-09-16T04:55:18.580014+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:18.580014+0000","last_undegraded":"2024-09-16T04:55:18.580014+0000","last_fullsized":"2024-09-16T04:55:18.580014+0000","mapping_epoch":46,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24530944,"num_objects":5989,"num_object_clones":0,"num_object_copies":11978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10712,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'13989","reported_seq":14092,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:18.623539+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:18.623539+0000","last_peered":"2024-09-16T04:55:18.623539+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:18.623539+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113769,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13988,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2481}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14409","reported_seq":14517,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:18.540976+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:18.540976+0000","last_peered":"2024-09-16T04:55:18.540976+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:18.540976+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25081659,"num_objects":6126,"num_object_clones":0,"num_object_copies":12252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14406,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2612}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14056","reported_seq":20855,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:18.045047+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:18.045047+0000","last_peered":"2024-09-16T04:55:18.045047+0000","last_clean":"2024-09-16T04:55:18.045047+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:18.045047+0000","last_undegraded":"2024-09-16T04:55:18.045047+0000","last_fullsized":"2024-09-16T04:55:18.045047+0000","mapping_epoch":48,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25157632,"num_objects":6142,"num_object_clones":0,"num_object_copies":12284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10810,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14393","reported_seq":14538,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:19.176336+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:19.176336+0000","last_peered":"2024-09-16T04:55:19.176336+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:19.176336+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24670825,"num_objects":6024,"num_object_clones":0,"num_object_copies":12048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11162,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2500}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":394371319,"num_objects":96289,"num_object_clones":0,"num_object_copies":192578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32682,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190341,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8060,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":956375040,"data_stored":956300795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681164,"kb_used_data":324604,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93690676,"statfs":{"total":96636764160,"available":95939252224,"internally_reserved":0,"allocated":332394496,"data_stored":332111170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593797,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619088,"kb_used_data":290300,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93752752,"statfs":{"total":96636764160,"available":96002818048,"internally_reserved":0,"allocated":297267200,"data_stored":296928210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674300,"kb_used_data":321956,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93697540,"statfs":{"total":96636764160,"available":95946280960,"internally_reserved":0,"allocated":329682944,"data_stored":329403690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":329097216,"data_stored":329093737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296271872,"data_stored":296208657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331005952,"data_stored":330998401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:24.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:55:24.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'13838", 'reported_seq': 13923, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:19.060043+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:55:19.060043+0000', 'last_peered': '2024-09-16T04:55:19.060043+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:55:19.060043+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'3800", 'ondisk_log_start': "45'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24166400, 'num_objects': 5900, 'num_object_clones': 0, 'num_object_copies': 11800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13838, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2559}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:24.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:55:24.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'13868", 'reported_seq': 14011, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:18.650893+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:55:18.650893+0000', 'last_peered': '2024-09-16T04:55:18.650893+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:55:18.650893+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'3800", 'ondisk_log_start': "45'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24530968, 'num_objects': 5990, 'num_object_clones': 0, 'num_object_copies': 11980, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5990, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10607, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2671}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:24.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:55:24.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'14242", 'reported_seq': 14381, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:19.130170+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:55:19.130170+0000', 'last_peered': '2024-09-16T04:55:19.130170+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:55:19.130170+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'4200", 'ondisk_log_start': "45'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24866816, 'num_objects': 6071, 'num_object_clones': 0, 'num_object_copies': 12142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6071, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11235, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2615}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:24.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:55:24.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'13990", 'reported_seq': 14144, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:19.111907+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:55:19.111907+0000', 'last_peered': '2024-09-16T04:55:19.111907+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:55:19.111907+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'3900", 'ondisk_log_start': "45'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24043520, 'num_objects': 5870, 'num_object_clones': 0, 'num_object_copies': 11740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5870, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10645, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2471}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:24.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:55:24.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'14015", 'reported_seq': 14248, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:18.573783+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:55:18.573783+0000', 'last_peered': '2024-09-16T04:55:18.573783+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:55:18.573783+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'4000", 'ondisk_log_start': "45'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24846336, 'num_objects': 6066, 'num_object_clones': 0, 'num_object_copies': 12132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6066, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10766, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2745}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:24.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:55:24.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'14214", 'reported_seq': 14349, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:19.129070+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:55:19.129070+0000', 'last_peered': '2024-09-16T04:55:19.129070+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:55:19.129070+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'4200", 'ondisk_log_start': "45'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24592384, 'num_objects': 6004, 'num_object_clones': 0, 'num_object_copies': 12008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10869, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2539}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:24.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:55:24.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'13965", 'reported_seq': 16969, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:18.649907+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:55:18.649907+0000', 'last_peered': '2024-09-16T04:55:18.649907+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:55:18.649907+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'3900", 'ondisk_log_start': "45'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24634578, 'num_objects': 6016, 'num_object_clones': 0, 'num_object_copies': 12032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13937, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1380, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 4096}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:24.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:55:24.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'13989", 'reported_seq': 14092, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:18.623539+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:55:18.623539+0000', 'last_peered': '2024-09-16T04:55:18.623539+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:55:18.623539+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'3900", 'ondisk_log_start': "45'3900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24113769, 'num_objects': 5888, 'num_object_clones': 0, 'num_object_copies': 11776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13988, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2481}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:24.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:55:24.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'14409", 'reported_seq': 14517, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:18.540976+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:55:18.540976+0000', 'last_peered': '2024-09-16T04:55:18.540976+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:55:18.540976+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'4400", 'ondisk_log_start': "45'4400", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25081659, 'num_objects': 6126, 'num_object_clones': 0, 'num_object_copies': 12252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14406, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2612}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:24.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:55:24.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'14393", 'reported_seq': 14538, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:19.176336+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:55:19.176336+0000', 'last_peered': '2024-09-16T04:55:19.176336+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:55:19.176336+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'4300", 'ondisk_log_start': "45'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24670825, 'num_objects': 6024, 'num_object_clones': 0, 'num_object_copies': 12048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11162, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2500}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:24.279 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:55:25.206 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:25.206 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32573/187870 objects degraded (17.338%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":472,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":93935,"data_bytes":384721162,"bytes_used":2006761472,"bytes_avail":287903531008,"bytes_total":289910292480,"degraded_objects":32573,"degraded_total":187870,"degraded_ratio":0.17338052908926385,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":302,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":585},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:55:25.267 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:55:25.267 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:26.119 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:26.121 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:26.177 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-16T04:55:24.423699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":384721162,"num_objects":93935,"num_object_clones":0,"num_object_copies":187870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32573,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93935,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192699,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8170,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160622,"ondisk_log_size":160622,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1959728,"kb_used_data":917884,"kb_used_omap":12,"kb_used_meta":1041779,"kb_avail":281155792,"statfs":{"total":289910292480,"available":287903531008,"internally_reserved":0,"allocated":939913216,"data_stored":939011646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1066781697},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,8,6,15,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28794880,"num_objects":-7030,"num_object_clones":0,"num_object_copies":-14060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7030,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002911"},"pg_stats":[{"pgid":"2.f","version":"49'13964","reported_seq":14048,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:24.161380+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:24.161380+0000","last_peered":"2024-09-16T04:55:24.161380+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:24.161380+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13964,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2433}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14164","reported_seq":14285,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:23.522539+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:23.522539+0000","last_peered":"2024-09-16T04:55:23.522539+0000","last_clean":"2024-09-16T04:55:23.522539+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:23.522539+0000","last_undegraded":"2024-09-16T04:55:23.522539+0000","last_fullsized":"2024-09-16T04:55:23.522539+0000","mapping_epoch":35,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23879680,"num_objects":5830,"num_object_clones":0,"num_object_copies":11660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11133,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14002","reported_seq":14146,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:23.595745+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:23.595745+0000","last_peered":"2024-09-16T04:55:23.595745+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:23.595745+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23982104,"num_objects":5856,"num_object_clones":0,"num_object_copies":11712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10741,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14358","reported_seq":14433,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:23.628429+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:23.628429+0000","last_peered":"2024-09-16T04:55:23.628429+0000","last_clean":"2024-09-16T04:55:23.628429+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:23.628429+0000","last_undegraded":"2024-09-16T04:55:23.628429+0000","last_fullsized":"2024-09-16T04:55:23.628429+0000","mapping_epoch":35,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24518656,"num_objects":5986,"num_object_clones":0,"num_object_copies":11972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14358,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'14266","reported_seq":14490,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:24.161711+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:24.161711+0000","last_peered":"2024-09-16T04:55:24.161711+0000","last_clean":"2024-09-16T04:55:24.161711+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:24.161711+0000","last_undegraded":"2024-09-16T04:55:24.161711+0000","last_fullsized":"2024-09-16T04:55:24.161711+0000","mapping_epoch":34,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24076288,"num_objects":5878,"num_object_clones":0,"num_object_copies":11756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11298,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14410","reported_seq":14549,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:24.125615+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:24.125615+0000","last_peered":"2024-09-16T04:55:24.125615+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:24.125615+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24178688,"num_objects":5903,"num_object_clones":0,"num_object_copies":11806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11403,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2447}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14125","reported_seq":14278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:24.176222+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:24.176222+0000","last_peered":"2024-09-16T04:55:24.176222+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:24.176222+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23490560,"num_objects":5735,"num_object_clones":0,"num_object_copies":11470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10780,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2336}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'14151","reported_seq":14372,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:24.176080+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:24.176080+0000","last_peered":"2024-09-16T04:55:24.176080+0000","last_clean":"2024-09-16T04:55:24.176080+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:24.176080+0000","last_undegraded":"2024-09-16T04:55:24.176080+0000","last_fullsized":"2024-09-16T04:55:24.176080+0000","mapping_epoch":34,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24023040,"num_objects":5865,"num_object_clones":0,"num_object_copies":11730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11221,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14157","reported_seq":14390,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:23.649124+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:23.649124+0000","last_peered":"2024-09-16T04:55:23.649124+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:23.649124+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264704,"num_objects":5924,"num_object_clones":0,"num_object_copies":11848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10908,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2603}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14368","reported_seq":14503,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:24.176354+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:24.176354+0000","last_peered":"2024-09-16T04:55:24.176354+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:24.176354+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23961600,"num_objects":5850,"num_object_clones":0,"num_object_copies":11700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11023,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2385}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14123","reported_seq":17344,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:23.653842+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:23.653842+0000","last_peered":"2024-09-16T04:55:23.653842+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:23.653842+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23987410,"num_objects":5858,"num_object_clones":0,"num_object_copies":11716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1811,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14095,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1489,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4047}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'14131","reported_seq":20993,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:23.692291+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:23.692291+0000","last_peered":"2024-09-16T04:55:23.692291+0000","last_clean":"2024-09-16T04:55:23.692291+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:23.692291+0000","last_undegraded":"2024-09-16T04:55:23.692291+0000","last_fullsized":"2024-09-16T04:55:23.692291+0000","mapping_epoch":46,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23949312,"num_objects":5847,"num_object_clones":0,"num_object_copies":11694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10854,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14123","reported_seq":14222,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:23.707160+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:23.707160+0000","last_peered":"2024-09-16T04:55:23.707160+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:23.707160+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23564905,"num_objects":5754,"num_object_clones":0,"num_object_copies":11508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14122,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2347}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14539","reported_seq":14647,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:23.665368+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:23.665368+0000","last_peered":"2024-09-16T04:55:23.665368+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:23.665368+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24549179,"num_objects":5996,"num_object_clones":0,"num_object_copies":11992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14536,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2482}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14223","reported_seq":21022,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:23.131999+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:23.131999+0000","last_peered":"2024-09-16T04:55:23.131999+0000","last_clean":"2024-09-16T04:55:23.131999+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:23.131999+0000","last_undegraded":"2024-09-16T04:55:23.131999+0000","last_fullsized":"2024-09-16T04:55:23.131999+0000","mapping_epoch":48,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24473600,"num_objects":5975,"num_object_clones":0,"num_object_copies":11950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10977,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14515","reported_seq":14660,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:24.066794+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:24.066794+0000","last_peered":"2024-09-16T04:55:24.066794+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:24.066794+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24171113,"num_objects":5902,"num_object_clones":0,"num_object_copies":11804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11284,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2378}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":384721143,"num_objects":93933,"num_object_clones":0,"num_object_copies":187866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32573,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93933,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192697,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8169,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":937050112,"data_stored":936975867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160619,"ondisk_log_size":160619,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679252,"kb_used_data":318528,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93692588,"statfs":{"total":96636764160,"available":95941210112,"internally_reserved":0,"allocated":326172672,"data_stored":325889346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593798,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603900,"kb_used_data":283436,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93767940,"statfs":{"total":96636764160,"available":96018370560,"internally_reserved":0,"allocated":290238464,"data_stored":289899474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,6,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676576,"kb_used_data":315920,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93695264,"statfs":{"total":96636764160,"available":95943950336,"internally_reserved":0,"allocated":323502080,"data_stored":323222826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322985984,"data_stored":322982505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289206272,"data_stored":289143057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324857856,"data_stored":324850305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:29.179 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:30.098 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:30.103 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:30.193 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-16T04:55:28.424796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":384176394,"num_objects":93802,"num_object_clones":0,"num_object_copies":187604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32573,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93802,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192832,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8170,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160655,"ondisk_log_size":160655,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1956988,"kb_used_data":910984,"kb_used_omap":12,"kb_used_meta":1045939,"kb_avail":281158532,"statfs":{"total":289910292480,"available":287906336768,"internally_reserved":0,"allocated":932847616,"data_stored":931946046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1071041537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,6,9,10,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19881984,"num_objects":-4854,"num_object_clones":0,"num_object_copies":-9708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4854,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002915"},"pg_stats":[{"pgid":"2.f","version":"49'13964","reported_seq":14048,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:24.161380+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:24.161380+0000","last_peered":"2024-09-16T04:55:24.161380+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:24.161380+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'3900","ondisk_log_start":"45'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23650304,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13964,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2433}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14164","reported_seq":14285,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:23.522539+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:23.522539+0000","last_peered":"2024-09-16T04:55:23.522539+0000","last_clean":"2024-09-16T04:55:23.522539+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:23.522539+0000","last_undegraded":"2024-09-16T04:55:23.522539+0000","last_fullsized":"2024-09-16T04:55:23.522539+0000","mapping_epoch":35,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23879680,"num_objects":5830,"num_object_clones":0,"num_object_copies":11660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11133,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14002","reported_seq":14146,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:23.595745+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:23.595745+0000","last_peered":"2024-09-16T04:55:23.595745+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:23.595745+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4000","ondisk_log_start":"45'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23982104,"num_objects":5856,"num_object_clones":0,"num_object_copies":11712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10741,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2537}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14358","reported_seq":14433,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:23.628429+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:23.628429+0000","last_peered":"2024-09-16T04:55:23.628429+0000","last_clean":"2024-09-16T04:55:23.628429+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:23.628429+0000","last_undegraded":"2024-09-16T04:55:23.628429+0000","last_fullsized":"2024-09-16T04:55:23.628429+0000","mapping_epoch":35,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24518656,"num_objects":5986,"num_object_clones":0,"num_object_copies":11972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14358,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'14266","reported_seq":14490,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:24.161711+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:24.161711+0000","last_peered":"2024-09-16T04:55:24.161711+0000","last_clean":"2024-09-16T04:55:24.161711+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:24.161711+0000","last_undegraded":"2024-09-16T04:55:24.161711+0000","last_fullsized":"2024-09-16T04:55:24.161711+0000","mapping_epoch":34,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24076288,"num_objects":5878,"num_object_clones":0,"num_object_copies":11756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11298,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14410","reported_seq":14549,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:24.125615+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:24.125615+0000","last_peered":"2024-09-16T04:55:24.125615+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:24.125615+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24178688,"num_objects":5903,"num_object_clones":0,"num_object_copies":11806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11403,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2447}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14125","reported_seq":14278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:24.176222+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:24.176222+0000","last_peered":"2024-09-16T04:55:24.176222+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:24.176222+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23490560,"num_objects":5735,"num_object_clones":0,"num_object_copies":11470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10780,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2336}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'14151","reported_seq":14372,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:24.176080+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:24.176080+0000","last_peered":"2024-09-16T04:55:24.176080+0000","last_clean":"2024-09-16T04:55:24.176080+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:24.176080+0000","last_undegraded":"2024-09-16T04:55:24.176080+0000","last_fullsized":"2024-09-16T04:55:24.176080+0000","mapping_epoch":34,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24023040,"num_objects":5865,"num_object_clones":0,"num_object_copies":11730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11221,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14157","reported_seq":14390,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:23.649124+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:23.649124+0000","last_peered":"2024-09-16T04:55:23.649124+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:23.649124+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24264704,"num_objects":5924,"num_object_clones":0,"num_object_copies":11848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10908,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2603}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14368","reported_seq":14503,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:24.176354+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:24.176354+0000","last_peered":"2024-09-16T04:55:24.176354+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:24.176354+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23961600,"num_objects":5850,"num_object_clones":0,"num_object_copies":11700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11023,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2385}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14123","reported_seq":17344,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:23.653842+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:23.653842+0000","last_peered":"2024-09-16T04:55:23.653842+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:23.653842+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23987410,"num_objects":5858,"num_object_clones":0,"num_object_copies":11716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1811,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14095,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1489,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":4047}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'14131","reported_seq":20993,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:23.692291+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:23.692291+0000","last_peered":"2024-09-16T04:55:23.692291+0000","last_clean":"2024-09-16T04:55:23.692291+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:23.692291+0000","last_undegraded":"2024-09-16T04:55:23.692291+0000","last_fullsized":"2024-09-16T04:55:23.692291+0000","mapping_epoch":46,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23949312,"num_objects":5847,"num_object_clones":0,"num_object_copies":11694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10854,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14123","reported_seq":14222,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:23.707160+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:23.707160+0000","last_peered":"2024-09-16T04:55:23.707160+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:23.707160+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23564905,"num_objects":5754,"num_object_clones":0,"num_object_copies":11508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14122,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2347}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14539","reported_seq":14647,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:23.665368+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:23.665368+0000","last_peered":"2024-09-16T04:55:23.665368+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:23.665368+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24549179,"num_objects":5996,"num_object_clones":0,"num_object_copies":11992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14536,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2482}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14356","reported_seq":21154,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:28.167210+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:28.167210+0000","last_peered":"2024-09-16T04:55:28.167210+0000","last_clean":"2024-09-16T04:55:28.167210+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:28.167210+0000","last_undegraded":"2024-09-16T04:55:28.167210+0000","last_fullsized":"2024-09-16T04:55:28.167210+0000","mapping_epoch":48,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23928832,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11110,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14515","reported_seq":14660,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:24.066794+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:24.066794+0000","last_peered":"2024-09-16T04:55:24.066794+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:24.066794+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24171113,"num_objects":5902,"num_object_clones":0,"num_object_copies":11804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11284,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2378}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":384176375,"num_objects":93800,"num_object_clones":0,"num_object_copies":187600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32573,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93800,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":192830,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8169,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":929812480,"data_stored":929738235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160652,"ondisk_log_size":160652,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679252,"kb_used_data":318528,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93692588,"statfs":{"total":96636764160,"available":95941210112,"internally_reserved":0,"allocated":326172672,"data_stored":325889346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593799,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601160,"kb_used_data":276536,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93770680,"statfs":{"total":96636764160,"available":96021176320,"internally_reserved":0,"allocated":283172864,"data_stored":282833874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,3,1,2,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676576,"kb_used_data":315920,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93695264,"statfs":{"total":96636764160,"available":95943950336,"internally_reserved":0,"allocated":323502080,"data_stored":323222826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322985984,"data_stored":322982505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":281968640,"data_stored":281905425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324857856,"data_stored":324850305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:30.194 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:30.983 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:30.984 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:31.067 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-09-16T04:55:30.425248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":375099658,"num_objects":91586,"num_object_clones":0,"num_object_copies":183172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91586,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195048,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8272,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1953660,"kb_used_data":899328,"kb_used_omap":12,"kb_used_meta":1054259,"kb_avail":281161860,"statfs":{"total":289910292480,"available":287909744640,"internally_reserved":0,"allocated":920911872,"data_stored":920010302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1079561217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,2,5,9,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28336128,"num_objects":-6918,"num_object_clones":0,"num_object_copies":-13836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6918,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002936"},"pg_stats":[{"pgid":"2.f","version":"49'14132","reported_seq":14217,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:29.150572+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:29.150572+0000","last_peered":"2024-09-16T04:55:29.150572+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:29.150572+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22962176,"num_objects":5606,"num_object_clones":0,"num_object_copies":11212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14132,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2265}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14276","reported_seq":14397,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:28.568770+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:28.568770+0000","last_peered":"2024-09-16T04:55:28.568770+0000","last_clean":"2024-09-16T04:55:28.568770+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:28.568770+0000","last_undegraded":"2024-09-16T04:55:28.568770+0000","last_fullsized":"2024-09-16T04:55:28.568770+0000","mapping_epoch":35,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11245,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14187","reported_seq":14329,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:28.753216+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:28.753216+0000","last_peered":"2024-09-16T04:55:28.753216+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:28.753216+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23224344,"num_objects":5671,"num_object_clones":0,"num_object_copies":11342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5671,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10926,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2352}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14503","reported_seq":14578,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:28.730706+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:28.730706+0000","last_peered":"2024-09-16T04:55:28.730706+0000","last_clean":"2024-09-16T04:55:28.730706+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:28.730706+0000","last_undegraded":"2024-09-16T04:55:28.730706+0000","last_fullsized":"2024-09-16T04:55:28.730706+0000","mapping_epoch":35,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23924736,"num_objects":5841,"num_object_clones":0,"num_object_copies":11682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14503,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'14409","reported_seq":14633,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:29.149236+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:29.149236+0000","last_peered":"2024-09-16T04:55:29.149236+0000","last_clean":"2024-09-16T04:55:29.149236+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:29.149236+0000","last_undegraded":"2024-09-16T04:55:29.149236+0000","last_fullsized":"2024-09-16T04:55:29.149236+0000","mapping_epoch":34,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23490560,"num_objects":5735,"num_object_clones":0,"num_object_copies":11470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11441,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14561","reported_seq":14694,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:29.142604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:29.142604+0000","last_peered":"2024-09-16T04:55:29.142604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:29.142604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23560192,"num_objects":5752,"num_object_clones":0,"num_object_copies":11504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11554,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2296}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14253","reported_seq":14407,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:29.189092+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:29.189092+0000","last_peered":"2024-09-16T04:55:29.189092+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:29.189092+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22966272,"num_objects":5607,"num_object_clones":0,"num_object_copies":11214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10908,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2208}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'14290","reported_seq":14511,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:29.142853+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:29.142853+0000","last_peered":"2024-09-16T04:55:29.142853+0000","last_clean":"2024-09-16T04:55:29.142853+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:29.142853+0000","last_undegraded":"2024-09-16T04:55:29.142853+0000","last_fullsized":"2024-09-16T04:55:29.142853+0000","mapping_epoch":34,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23453696,"num_objects":5726,"num_object_clones":0,"num_object_copies":11452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11360,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14316","reported_seq":14549,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:28.661530+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:28.661530+0000","last_peered":"2024-09-16T04:55:28.661530+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:28.661530+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23613440,"num_objects":5765,"num_object_clones":0,"num_object_copies":11530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11067,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2444}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14531","reported_seq":14664,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:29.179585+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:29.179585+0000","last_peered":"2024-09-16T04:55:29.179585+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:29.179585+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23293952,"num_objects":5687,"num_object_clones":0,"num_object_copies":11374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11186,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14284","reported_seq":17707,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:28.761592+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:28.761592+0000","last_peered":"2024-09-16T04:55:28.761592+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:28.761592+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23327954,"num_objects":5697,"num_object_clones":0,"num_object_copies":11394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1591,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3988}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'14275","reported_seq":21137,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:28.741572+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:28.741572+0000","last_peered":"2024-09-16T04:55:28.741572+0000","last_clean":"2024-09-16T04:55:28.741572+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:28.741572+0000","last_undegraded":"2024-09-16T04:55:28.741572+0000","last_fullsized":"2024-09-16T04:55:28.741572+0000","mapping_epoch":46,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23359488,"num_objects":5703,"num_object_clones":0,"num_object_copies":11406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10998,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14251","reported_seq":14354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:28.617285+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:28.617285+0000","last_peered":"2024-09-16T04:55:28.617285+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:28.617285+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23040617,"num_objects":5626,"num_object_clones":0,"num_object_copies":11252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14250,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2219}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14682","reported_seq":14790,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:28.677021+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:28.677021+0000","last_peered":"2024-09-16T04:55:28.677021+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:28.677021+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23963451,"num_objects":5853,"num_object_clones":0,"num_object_copies":11706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14679,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2339}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14356","reported_seq":21154,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:28.167210+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:28.167210+0000","last_peered":"2024-09-16T04:55:28.167210+0000","last_clean":"2024-09-16T04:55:28.167210+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:28.167210+0000","last_undegraded":"2024-09-16T04:55:28.167210+0000","last_fullsized":"2024-09-16T04:55:28.167210+0000","mapping_epoch":48,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23928832,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11110,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14662","reported_seq":14807,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:29.212799+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:29.212799+0000","last_peered":"2024-09-16T04:55:29.212799+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:29.212799+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23569001,"num_objects":5755,"num_object_clones":0,"num_object_copies":11510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11431,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":375099639,"num_objects":91584,"num_object_clones":0,"num_object_copies":183168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91584,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195046,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8271,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":917991424,"data_stored":917917179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677584,"kb_used_data":312704,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93694256,"statfs":{"total":96636764160,"available":95942918144,"internally_reserved":0,"allocated":320208896,"data_stored":319925570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593799,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601160,"kb_used_data":276536,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93770680,"statfs":{"total":96636764160,"available":96021176320,"internally_reserved":0,"allocated":283172864,"data_stored":282833874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,3,1,2,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674916,"kb_used_data":310088,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93696924,"statfs":{"total":96636764160,"available":95945650176,"internally_reserved":0,"allocated":317530112,"data_stored":317250858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":317087744,"data_stored":317084265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":281968640,"data_stored":281905425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":318935040,"data_stored":318927489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:31.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:55:31.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'14132", 'reported_seq': 14217, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:29.150572+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:55:29.150572+0000', 'last_peered': '2024-09-16T04:55:29.150572+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:55:29.150572+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'4100", 'ondisk_log_start': "45'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22962176, 'num_objects': 5606, 'num_object_clones': 0, 'num_object_copies': 11212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5606, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14132, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2265}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:31.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:55:31.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'14187", 'reported_seq': 14329, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:28.753216+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:55:28.753216+0000', 'last_peered': '2024-09-16T04:55:28.753216+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:55:28.753216+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'4100", 'ondisk_log_start': "45'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23224344, 'num_objects': 5671, 'num_object_clones': 0, 'num_object_copies': 11342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5671, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10926, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2352}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:31.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:55:31.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'14561", 'reported_seq': 14694, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:29.142604+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:55:29.142604+0000', 'last_peered': '2024-09-16T04:55:29.142604+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:55:29.142604+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'4500", 'ondisk_log_start': "45'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23560192, 'num_objects': 5752, 'num_object_clones': 0, 'num_object_copies': 11504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5752, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11554, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2296}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:31.069 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:55:31.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'14253", 'reported_seq': 14407, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:29.189092+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:55:29.189092+0000', 'last_peered': '2024-09-16T04:55:29.189092+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:55:29.189092+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'4200", 'ondisk_log_start': "45'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22966272, 'num_objects': 5607, 'num_object_clones': 0, 'num_object_copies': 11214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5607, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10908, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2208}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:31.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:55:31.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'14316", 'reported_seq': 14549, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:28.661530+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:55:28.661530+0000', 'last_peered': '2024-09-16T04:55:28.661530+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:55:28.661530+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'4300", 'ondisk_log_start': "45'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23613440, 'num_objects': 5765, 'num_object_clones': 0, 'num_object_copies': 11530, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5765, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11067, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2444}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:31.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:55:31.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'14531", 'reported_seq': 14664, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:29.179585+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:55:29.179585+0000', 'last_peered': '2024-09-16T04:55:29.179585+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:55:29.179585+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'4500", 'ondisk_log_start': "45'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23293952, 'num_objects': 5687, 'num_object_clones': 0, 'num_object_copies': 11374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5687, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11186, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2222}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:31.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:55:31.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'14284", 'reported_seq': 17707, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:28.761592+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:55:28.761592+0000', 'last_peered': '2024-09-16T04:55:28.761592+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:55:28.761592+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'4200", 'ondisk_log_start': "45'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23327954, 'num_objects': 5697, 'num_object_clones': 0, 'num_object_copies': 11394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1709, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14256, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1591, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3988}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:31.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:55:31.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'14251", 'reported_seq': 14354, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:28.617285+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:55:28.617285+0000', 'last_peered': '2024-09-16T04:55:28.617285+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:55:28.617285+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'4200", 'ondisk_log_start': "45'4200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23040617, 'num_objects': 5626, 'num_object_clones': 0, 'num_object_copies': 11252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14250, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2219}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:31.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:55:31.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'14682", 'reported_seq': 14790, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:28.677021+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:55:28.677021+0000', 'last_peered': '2024-09-16T04:55:28.677021+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:55:28.677021+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'4600", 'ondisk_log_start': "45'4600", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23963451, 'num_objects': 5853, 'num_object_clones': 0, 'num_object_copies': 11706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14679, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2339}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:31.071 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:55:31.071 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'14662", 'reported_seq': 14807, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:29.212799+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:55:29.212799+0000', 'last_peered': '2024-09-16T04:55:29.212799+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:55:29.212799+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'4600", 'ondisk_log_start': "45'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23569001, 'num_objects': 5755, 'num_object_clones': 0, 'num_object_copies': 11510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5755, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11431, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2231}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:31.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:55:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:31.978 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32471/183172 objects degraded (17.727%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":479,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":91586,"data_bytes":375099658,"bytes_used":2000547840,"bytes_avail":287909744640,"bytes_total":289910292480,"degraded_objects":32471,"degraded_total":183172,"degraded_ratio":0.1772705435328544,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":302,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":576},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:55:32.012 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:55:32.013 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:32.687 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:32.689 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:32.776 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-16T04:55:32.425740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":375099658,"num_objects":91586,"num_object_clones":0,"num_object_copies":183172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91586,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195048,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8272,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1953660,"kb_used_data":899328,"kb_used_omap":12,"kb_used_meta":1054259,"kb_avail":281161860,"statfs":{"total":289910292480,"available":287909744640,"internally_reserved":0,"allocated":920911872,"data_stored":920010302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1079561217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,2,5,9,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19271680,"num_objects":-4705,"num_object_clones":0,"num_object_copies":-9410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4705,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002982"},"pg_stats":[{"pgid":"2.f","version":"49'14132","reported_seq":14217,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:29.150572+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:29.150572+0000","last_peered":"2024-09-16T04:55:29.150572+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:29.150572+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22962176,"num_objects":5606,"num_object_clones":0,"num_object_copies":11212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14132,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2265}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14276","reported_seq":14397,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:28.568770+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:28.568770+0000","last_peered":"2024-09-16T04:55:28.568770+0000","last_clean":"2024-09-16T04:55:28.568770+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:28.568770+0000","last_undegraded":"2024-09-16T04:55:28.568770+0000","last_fullsized":"2024-09-16T04:55:28.568770+0000","mapping_epoch":35,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11245,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14187","reported_seq":14329,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:28.753216+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:28.753216+0000","last_peered":"2024-09-16T04:55:28.753216+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:28.753216+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4100","ondisk_log_start":"45'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23224344,"num_objects":5671,"num_object_clones":0,"num_object_copies":11342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5671,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10926,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2352}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14503","reported_seq":14578,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:28.730706+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:28.730706+0000","last_peered":"2024-09-16T04:55:28.730706+0000","last_clean":"2024-09-16T04:55:28.730706+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:28.730706+0000","last_undegraded":"2024-09-16T04:55:28.730706+0000","last_fullsized":"2024-09-16T04:55:28.730706+0000","mapping_epoch":35,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23924736,"num_objects":5841,"num_object_clones":0,"num_object_copies":11682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14503,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'14409","reported_seq":14633,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:29.149236+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:29.149236+0000","last_peered":"2024-09-16T04:55:29.149236+0000","last_clean":"2024-09-16T04:55:29.149236+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:29.149236+0000","last_undegraded":"2024-09-16T04:55:29.149236+0000","last_fullsized":"2024-09-16T04:55:29.149236+0000","mapping_epoch":34,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23490560,"num_objects":5735,"num_object_clones":0,"num_object_copies":11470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11441,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14561","reported_seq":14694,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:29.142604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:29.142604+0000","last_peered":"2024-09-16T04:55:29.142604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:29.142604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23560192,"num_objects":5752,"num_object_clones":0,"num_object_copies":11504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11554,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2296}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14253","reported_seq":14407,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:29.189092+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:29.189092+0000","last_peered":"2024-09-16T04:55:29.189092+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:29.189092+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22966272,"num_objects":5607,"num_object_clones":0,"num_object_copies":11214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10908,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2208}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'14290","reported_seq":14511,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:29.142853+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:29.142853+0000","last_peered":"2024-09-16T04:55:29.142853+0000","last_clean":"2024-09-16T04:55:29.142853+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:29.142853+0000","last_undegraded":"2024-09-16T04:55:29.142853+0000","last_fullsized":"2024-09-16T04:55:29.142853+0000","mapping_epoch":34,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23453696,"num_objects":5726,"num_object_clones":0,"num_object_copies":11452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11360,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14316","reported_seq":14549,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:28.661530+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:28.661530+0000","last_peered":"2024-09-16T04:55:28.661530+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:28.661530+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23613440,"num_objects":5765,"num_object_clones":0,"num_object_copies":11530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11067,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2444}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14531","reported_seq":14664,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:29.179585+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:29.179585+0000","last_peered":"2024-09-16T04:55:29.179585+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:29.179585+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23293952,"num_objects":5687,"num_object_clones":0,"num_object_copies":11374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11186,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14284","reported_seq":17707,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:28.761592+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:28.761592+0000","last_peered":"2024-09-16T04:55:28.761592+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:28.761592+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23327954,"num_objects":5697,"num_object_clones":0,"num_object_copies":11394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1591,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3988}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'14275","reported_seq":21137,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:28.741572+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:28.741572+0000","last_peered":"2024-09-16T04:55:28.741572+0000","last_clean":"2024-09-16T04:55:28.741572+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:28.741572+0000","last_undegraded":"2024-09-16T04:55:28.741572+0000","last_fullsized":"2024-09-16T04:55:28.741572+0000","mapping_epoch":46,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23359488,"num_objects":5703,"num_object_clones":0,"num_object_copies":11406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10998,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14251","reported_seq":14354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:28.617285+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:28.617285+0000","last_peered":"2024-09-16T04:55:28.617285+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:28.617285+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23040617,"num_objects":5626,"num_object_clones":0,"num_object_copies":11252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14250,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2219}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14682","reported_seq":14790,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:28.677021+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:28.677021+0000","last_peered":"2024-09-16T04:55:28.677021+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:28.677021+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23963451,"num_objects":5853,"num_object_clones":0,"num_object_copies":11706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14679,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2339}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14356","reported_seq":21154,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:28.167210+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:28.167210+0000","last_peered":"2024-09-16T04:55:28.167210+0000","last_clean":"2024-09-16T04:55:28.167210+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:28.167210+0000","last_undegraded":"2024-09-16T04:55:28.167210+0000","last_fullsized":"2024-09-16T04:55:28.167210+0000","mapping_epoch":48,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23928832,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11110,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14662","reported_seq":14807,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:29.212799+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:29.212799+0000","last_peered":"2024-09-16T04:55:29.212799+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:29.212799+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23569001,"num_objects":5755,"num_object_clones":0,"num_object_copies":11510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11431,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2231}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":375099639,"num_objects":91584,"num_object_clones":0,"num_object_copies":183168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91584,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195046,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8271,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":917991424,"data_stored":917917179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677584,"kb_used_data":312704,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93694256,"statfs":{"total":96636764160,"available":95942918144,"internally_reserved":0,"allocated":320208896,"data_stored":319925570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593799,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601160,"kb_used_data":276536,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93770680,"statfs":{"total":96636764160,"available":96021176320,"internally_reserved":0,"allocated":283172864,"data_stored":282833874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,3,1,2,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674916,"kb_used_data":310088,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93696924,"statfs":{"total":96636764160,"available":95945650176,"internally_reserved":0,"allocated":317530112,"data_stored":317250858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":317087744,"data_stored":317084265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":281968640,"data_stored":281905425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":318935040,"data_stored":318927489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:35.779 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:36.687 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:36.688 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:36.720 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-16T04:55:36.426654+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":365682954,"num_objects":89287,"num_object_clones":0,"num_object_copies":178574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89287,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197347,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8371,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1923988,"kb_used_data":882148,"kb_used_omap":12,"kb_used_meta":1041779,"kb_avail":281191532,"statfs":{"total":289910292480,"available":287940128768,"internally_reserved":0,"allocated":903319552,"data_stored":902417982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1066781697},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,2,5,7,10,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19038208,"num_objects":-4648,"num_object_clones":0,"num_object_copies":-9296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4648,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002954"},"pg_stats":[{"pgid":"2.f","version":"49'14260","reported_seq":14345,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.161608+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:34.161608+0000","last_peered":"2024-09-16T04:55:34.161608+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:34.161608+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22437888,"num_objects":5478,"num_object_clones":0,"num_object_copies":10956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14260,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2137}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14426","reported_seq":14547,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:33.745558+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:33.745558+0000","last_peered":"2024-09-16T04:55:33.745558+0000","last_clean":"2024-09-16T04:55:33.745558+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:33.745558+0000","last_undegraded":"2024-09-16T04:55:33.745558+0000","last_fullsized":"2024-09-16T04:55:33.745558+0000","mapping_epoch":35,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22806528,"num_objects":5568,"num_object_clones":0,"num_object_copies":11136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11395,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14315","reported_seq":14459,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.681932+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:33.681932+0000","last_peered":"2024-09-16T04:55:33.681932+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:33.681932+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22700056,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11054,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2224}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14661","reported_seq":14736,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:33.768440+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:33.768440+0000","last_peered":"2024-09-16T04:55:33.768440+0000","last_clean":"2024-09-16T04:55:33.768440+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:33.768440+0000","last_undegraded":"2024-09-16T04:55:33.768440+0000","last_fullsized":"2024-09-16T04:55:33.768440+0000","mapping_epoch":35,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23277568,"num_objects":5683,"num_object_clones":0,"num_object_copies":11366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14661,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'14552","reported_seq":14776,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:34.207683+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:34.207683+0000","last_peered":"2024-09-16T04:55:34.207683+0000","last_clean":"2024-09-16T04:55:34.207683+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:34.207683+0000","last_undegraded":"2024-09-16T04:55:34.207683+0000","last_fullsized":"2024-09-16T04:55:34.207683+0000","mapping_epoch":34,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22904832,"num_objects":5592,"num_object_clones":0,"num_object_copies":11184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11584,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14702","reported_seq":14841,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.188749+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:34.188749+0000","last_peered":"2024-09-16T04:55:34.188749+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:34.188749+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22982656,"num_objects":5611,"num_object_clones":0,"num_object_copies":11222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11695,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2155}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14402","reported_seq":14556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.141450+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:34.141450+0000","last_peered":"2024-09-16T04:55:34.141450+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:34.141450+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22355968,"num_objects":5458,"num_object_clones":0,"num_object_copies":10916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11057,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2059}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'14446","reported_seq":14667,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:34.167452+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:34.167452+0000","last_peered":"2024-09-16T04:55:34.167452+0000","last_clean":"2024-09-16T04:55:34.167452+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:34.167452+0000","last_undegraded":"2024-09-16T04:55:34.167452+0000","last_fullsized":"2024-09-16T04:55:34.167452+0000","mapping_epoch":34,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22814720,"num_objects":5570,"num_object_clones":0,"num_object_copies":11140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11516,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14482","reported_seq":14715,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.736996+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:33.736996+0000","last_peered":"2024-09-16T04:55:33.736996+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:33.736996+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22933504,"num_objects":5599,"num_object_clones":0,"num_object_copies":11198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11233,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2278}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14663","reported_seq":14798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.166367+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:34.166367+0000","last_peered":"2024-09-16T04:55:34.166367+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:34.166367+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22753280,"num_objects":5555,"num_object_clones":0,"num_object_copies":11110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11318,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2090}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14418","reported_seq":18042,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.756659+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:33.756659+0000","last_peered":"2024-09-16T04:55:33.756659+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:33.756659+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22779090,"num_objects":5563,"num_object_clones":0,"num_object_copies":11126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14390,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1690,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3953}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'14428","reported_seq":21290,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:33.753172+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:33.753172+0000","last_peered":"2024-09-16T04:55:33.753172+0000","last_clean":"2024-09-16T04:55:33.753172+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:33.753172+0000","last_undegraded":"2024-09-16T04:55:33.753172+0000","last_fullsized":"2024-09-16T04:55:33.753172+0000","mapping_epoch":46,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22732800,"num_objects":5550,"num_object_clones":0,"num_object_copies":11100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11151,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14394","reported_seq":14497,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.754573+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:33.754573+0000","last_peered":"2024-09-16T04:55:33.754573+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:33.754573+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22454889,"num_objects":5483,"num_object_clones":0,"num_object_copies":10966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14393,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2076}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14832","reported_seq":14938,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.693605+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:33.693605+0000","last_peered":"2024-09-16T04:55:33.693605+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:33.693605+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23349051,"num_objects":5703,"num_object_clones":0,"num_object_copies":11406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14829,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2189}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14480","reported_seq":21279,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:33.179959+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:33.179959+0000","last_peered":"2024-09-16T04:55:33.179959+0000","last_clean":"2024-09-16T04:55:33.179959+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:33.179959+0000","last_undegraded":"2024-09-16T04:55:33.179959+0000","last_fullsized":"2024-09-16T04:55:33.179959+0000","mapping_epoch":48,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11234,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14806","reported_seq":14947,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.220601+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:34.220601+0000","last_peered":"2024-09-16T04:55:34.220601+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:34.220601+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22979177,"num_objects":5611,"num_object_clones":0,"num_object_copies":11222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11575,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":365682935,"num_objects":89285,"num_object_clones":0,"num_object_copies":178570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89285,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197345,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8370,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":899006464,"data_stored":898932219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663464,"kb_used_data":306904,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93708376,"statfs":{"total":96636764160,"available":95957377024,"internally_reserved":0,"allocated":314269696,"data_stored":313986370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593800,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602764,"kb_used_data":269820,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93769076,"statfs":{"total":96636764160,"available":96019533824,"internally_reserved":0,"allocated":276295680,"data_stored":275956690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":340913838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":657760,"kb_used_data":305424,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93714080,"statfs":{"total":96636764160,"available":95963217920,"internally_reserved":0,"allocated":312754176,"data_stored":312474922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,0,3,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":311185408,"data_stored":311181929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274964480,"data_stored":274901265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312856576,"data_stored":312849025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:36.721 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:37.637 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:37.639 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:37.702 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-16T04:55:36.426654+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":365682954,"num_objects":89287,"num_object_clones":0,"num_object_copies":178574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89287,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197347,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8371,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1923988,"kb_used_data":882148,"kb_used_omap":12,"kb_used_meta":1041779,"kb_avail":281191532,"statfs":{"total":289910292480,"available":287940128768,"internally_reserved":0,"allocated":903319552,"data_stored":902417982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1066781697},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,2,5,7,10,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19038208,"num_objects":-4648,"num_object_clones":0,"num_object_copies":-9296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4648,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002954"},"pg_stats":[{"pgid":"2.f","version":"49'14260","reported_seq":14345,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.161608+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:34.161608+0000","last_peered":"2024-09-16T04:55:34.161608+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:34.161608+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22437888,"num_objects":5478,"num_object_clones":0,"num_object_copies":10956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14260,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2137}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14426","reported_seq":14547,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:33.745558+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:33.745558+0000","last_peered":"2024-09-16T04:55:33.745558+0000","last_clean":"2024-09-16T04:55:33.745558+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:33.745558+0000","last_undegraded":"2024-09-16T04:55:33.745558+0000","last_fullsized":"2024-09-16T04:55:33.745558+0000","mapping_epoch":35,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22806528,"num_objects":5568,"num_object_clones":0,"num_object_copies":11136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11395,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14315","reported_seq":14459,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.681932+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:33.681932+0000","last_peered":"2024-09-16T04:55:33.681932+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:33.681932+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22700056,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11054,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2224}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14661","reported_seq":14736,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:33.768440+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:33.768440+0000","last_peered":"2024-09-16T04:55:33.768440+0000","last_clean":"2024-09-16T04:55:33.768440+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:33.768440+0000","last_undegraded":"2024-09-16T04:55:33.768440+0000","last_fullsized":"2024-09-16T04:55:33.768440+0000","mapping_epoch":35,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23277568,"num_objects":5683,"num_object_clones":0,"num_object_copies":11366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14661,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'14552","reported_seq":14776,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:34.207683+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:34.207683+0000","last_peered":"2024-09-16T04:55:34.207683+0000","last_clean":"2024-09-16T04:55:34.207683+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:34.207683+0000","last_undegraded":"2024-09-16T04:55:34.207683+0000","last_fullsized":"2024-09-16T04:55:34.207683+0000","mapping_epoch":34,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22904832,"num_objects":5592,"num_object_clones":0,"num_object_copies":11184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11584,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14702","reported_seq":14841,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.188749+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:34.188749+0000","last_peered":"2024-09-16T04:55:34.188749+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:34.188749+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22982656,"num_objects":5611,"num_object_clones":0,"num_object_copies":11222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11695,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2155}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14402","reported_seq":14556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.141450+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:34.141450+0000","last_peered":"2024-09-16T04:55:34.141450+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:34.141450+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22355968,"num_objects":5458,"num_object_clones":0,"num_object_copies":10916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11057,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2059}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'14446","reported_seq":14667,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:34.167452+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:34.167452+0000","last_peered":"2024-09-16T04:55:34.167452+0000","last_clean":"2024-09-16T04:55:34.167452+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:34.167452+0000","last_undegraded":"2024-09-16T04:55:34.167452+0000","last_fullsized":"2024-09-16T04:55:34.167452+0000","mapping_epoch":34,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22814720,"num_objects":5570,"num_object_clones":0,"num_object_copies":11140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11516,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555825+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555825+0000","last_peered":"2024-09-16T04:54:05.555825+0000","last_clean":"2024-09-16T04:54:05.555825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555825+0000","last_undegraded":"2024-09-16T04:54:05.555825+0000","last_fullsized":"2024-09-16T04:54:05.555825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14482","reported_seq":14715,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.736996+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:33.736996+0000","last_peered":"2024-09-16T04:55:33.736996+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:33.736996+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22933504,"num_objects":5599,"num_object_clones":0,"num_object_copies":11198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11233,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2278}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14663","reported_seq":14798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.166367+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:34.166367+0000","last_peered":"2024-09-16T04:55:34.166367+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:34.166367+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22753280,"num_objects":5555,"num_object_clones":0,"num_object_copies":11110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11318,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2090}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14418","reported_seq":18042,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.756659+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:33.756659+0000","last_peered":"2024-09-16T04:55:33.756659+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:33.756659+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22779090,"num_objects":5563,"num_object_clones":0,"num_object_copies":11126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14390,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1690,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3953}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'14428","reported_seq":21290,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:33.753172+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:33.753172+0000","last_peered":"2024-09-16T04:55:33.753172+0000","last_clean":"2024-09-16T04:55:33.753172+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:33.753172+0000","last_undegraded":"2024-09-16T04:55:33.753172+0000","last_fullsized":"2024-09-16T04:55:33.753172+0000","mapping_epoch":46,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22732800,"num_objects":5550,"num_object_clones":0,"num_object_copies":11100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11151,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14394","reported_seq":14497,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.754573+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:33.754573+0000","last_peered":"2024-09-16T04:55:33.754573+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:33.754573+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22454889,"num_objects":5483,"num_object_clones":0,"num_object_copies":10966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14393,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2076}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14832","reported_seq":14938,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.693605+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:33.693605+0000","last_peered":"2024-09-16T04:55:33.693605+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:33.693605+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23349051,"num_objects":5703,"num_object_clones":0,"num_object_copies":11406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14829,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2189}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14480","reported_seq":21279,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:33.179959+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:33.179959+0000","last_peered":"2024-09-16T04:55:33.179959+0000","last_clean":"2024-09-16T04:55:33.179959+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:33.179959+0000","last_undegraded":"2024-09-16T04:55:33.179959+0000","last_fullsized":"2024-09-16T04:55:33.179959+0000","mapping_epoch":48,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11234,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14806","reported_seq":14947,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.220601+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:34.220601+0000","last_peered":"2024-09-16T04:55:34.220601+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:34.220601+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22979177,"num_objects":5611,"num_object_clones":0,"num_object_copies":11222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11575,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":365682935,"num_objects":89285,"num_object_clones":0,"num_object_copies":178570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89285,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197345,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8370,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":899006464,"data_stored":898932219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663464,"kb_used_data":306904,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93708376,"statfs":{"total":96636764160,"available":95957377024,"internally_reserved":0,"allocated":314269696,"data_stored":313986370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593800,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602764,"kb_used_data":269820,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93769076,"statfs":{"total":96636764160,"available":96019533824,"internally_reserved":0,"allocated":276295680,"data_stored":275956690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":340913838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":657760,"kb_used_data":305424,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93714080,"statfs":{"total":96636764160,"available":95963217920,"internally_reserved":0,"allocated":312754176,"data_stored":312474922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,0,3,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":311185408,"data_stored":311181929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274964480,"data_stored":274901265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312856576,"data_stored":312849025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:37.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:55:37.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'14260", 'reported_seq': 14345, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:34.161608+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:55:34.161608+0000', 'last_peered': '2024-09-16T04:55:34.161608+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:55:34.161608+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'4200", 'ondisk_log_start': "45'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22437888, 'num_objects': 5478, 'num_object_clones': 0, 'num_object_copies': 10956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5478, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14260, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2137}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:37.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:55:37.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'14315", 'reported_seq': 14459, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:33.681932+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:55:33.681932+0000', 'last_peered': '2024-09-16T04:55:33.681932+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:55:33.681932+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'4300", 'ondisk_log_start': "45'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22700056, 'num_objects': 5543, 'num_object_clones': 0, 'num_object_copies': 11086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5543, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11054, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2224}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:37.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:55:37.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'14702", 'reported_seq': 14841, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:34.188749+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:55:34.188749+0000', 'last_peered': '2024-09-16T04:55:34.188749+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:55:34.188749+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'4700", 'ondisk_log_start': "45'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22982656, 'num_objects': 5611, 'num_object_clones': 0, 'num_object_copies': 11222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5611, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11695, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2155}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:37.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:55:37.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'14402", 'reported_seq': 14556, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:34.141450+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:55:34.141450+0000', 'last_peered': '2024-09-16T04:55:34.141450+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:55:34.141450+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'4400", 'ondisk_log_start': "45'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22355968, 'num_objects': 5458, 'num_object_clones': 0, 'num_object_copies': 10916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5458, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11057, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2059}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:37.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:55:37.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'14482", 'reported_seq': 14715, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:33.736996+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:55:33.736996+0000', 'last_peered': '2024-09-16T04:55:33.736996+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:55:33.736996+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'4400", 'ondisk_log_start': "45'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22933504, 'num_objects': 5599, 'num_object_clones': 0, 'num_object_copies': 11198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5599, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11233, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2278}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:37.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:55:37.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'14663", 'reported_seq': 14798, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:34.166367+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:55:34.166367+0000', 'last_peered': '2024-09-16T04:55:34.166367+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:55:34.166367+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'4600", 'ondisk_log_start': "45'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22753280, 'num_objects': 5555, 'num_object_clones': 0, 'num_object_copies': 11110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5555, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11318, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2090}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:37.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:55:37.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'14418", 'reported_seq': 18042, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:33.756659+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:55:33.756659+0000', 'last_peered': '2024-09-16T04:55:33.756659+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:55:33.756659+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'4400", 'ondisk_log_start': "45'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22779090, 'num_objects': 5563, 'num_object_clones': 0, 'num_object_copies': 11126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1610, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5563, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14390, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1690, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3953}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:37.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:55:37.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'14394", 'reported_seq': 14497, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:33.754573+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:55:33.754573+0000', 'last_peered': '2024-09-16T04:55:33.754573+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:55:33.754573+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'4300", 'ondisk_log_start': "45'4300", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22454889, 'num_objects': 5483, 'num_object_clones': 0, 'num_object_copies': 10966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14393, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:37.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:55:37.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'14832", 'reported_seq': 14938, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:33.693605+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:55:33.693605+0000', 'last_peered': '2024-09-16T04:55:33.693605+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:55:33.693605+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'4800", 'ondisk_log_start': "45'4800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23349051, 'num_objects': 5703, 'num_object_clones': 0, 'num_object_copies': 11406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5703, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14829, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2189}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:37.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:55:37.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'14806", 'reported_seq': 14947, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:34.220601+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:55:34.220601+0000', 'last_peered': '2024-09-16T04:55:34.220601+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:55:34.220601+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'4800", 'ondisk_log_start': "45'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22979177, 'num_objects': 5611, 'num_object_clones': 0, 'num_object_copies': 11222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5611, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11575, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2087}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:37.705 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:55:38.724 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:38.725 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32372/178258 objects degraded (18.160%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":486,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":89129,"data_bytes":365035786,"bytes_used":1950134272,"bytes_avail":287960158208,"bytes_total":289910292480,"degraded_objects":32372,"degraded_total":178258,"degraded_ratio":0.18160194773867089,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":400},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:55:38.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:55:38.787 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:39.594 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:39.596 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:39.648 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-16T04:55:38.427067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":365035786,"num_objects":89129,"num_object_clones":0,"num_object_copies":178258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89129,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197505,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8371,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160628,"ondisk_log_size":160628,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1904428,"kb_used_data":875064,"kb_used_omap":12,"kb_used_meta":1029299,"kb_avail":281211092,"statfs":{"total":289910292480,"available":287960158208,"internally_reserved":0,"allocated":896065536,"data_stored":895163966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1054002177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,2,7,6,12,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19685376,"num_objects":-4806,"num_object_clones":0,"num_object_copies":-9612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4806,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002863"},"pg_stats":[{"pgid":"2.f","version":"49'14260","reported_seq":14345,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.161608+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:34.161608+0000","last_peered":"2024-09-16T04:55:34.161608+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:34.161608+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4200","ondisk_log_start":"45'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22437888,"num_objects":5478,"num_object_clones":0,"num_object_copies":10956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14260,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2137}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14426","reported_seq":14547,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:33.745558+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:33.745558+0000","last_peered":"2024-09-16T04:55:33.745558+0000","last_clean":"2024-09-16T04:55:33.745558+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:33.745558+0000","last_undegraded":"2024-09-16T04:55:33.745558+0000","last_fullsized":"2024-09-16T04:55:33.745558+0000","mapping_epoch":35,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22806528,"num_objects":5568,"num_object_clones":0,"num_object_copies":11136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11395,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14315","reported_seq":14459,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.681932+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:33.681932+0000","last_peered":"2024-09-16T04:55:33.681932+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:33.681932+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22700056,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11054,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2224}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14661","reported_seq":14736,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:33.768440+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:33.768440+0000","last_peered":"2024-09-16T04:55:33.768440+0000","last_clean":"2024-09-16T04:55:33.768440+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:33.768440+0000","last_undegraded":"2024-09-16T04:55:33.768440+0000","last_fullsized":"2024-09-16T04:55:33.768440+0000","mapping_epoch":35,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23277568,"num_objects":5683,"num_object_clones":0,"num_object_copies":11366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14661,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'14552","reported_seq":14776,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:34.207683+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:34.207683+0000","last_peered":"2024-09-16T04:55:34.207683+0000","last_clean":"2024-09-16T04:55:34.207683+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:34.207683+0000","last_undegraded":"2024-09-16T04:55:34.207683+0000","last_fullsized":"2024-09-16T04:55:34.207683+0000","mapping_epoch":34,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22904832,"num_objects":5592,"num_object_clones":0,"num_object_copies":11184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11584,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14702","reported_seq":14841,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.188749+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:34.188749+0000","last_peered":"2024-09-16T04:55:34.188749+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:34.188749+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22982656,"num_objects":5611,"num_object_clones":0,"num_object_copies":11222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11695,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2155}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14402","reported_seq":14556,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.141450+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:34.141450+0000","last_peered":"2024-09-16T04:55:34.141450+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:34.141450+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22355968,"num_objects":5458,"num_object_clones":0,"num_object_copies":10916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11057,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2059}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'14446","reported_seq":14667,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:34.167452+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:34.167452+0000","last_peered":"2024-09-16T04:55:34.167452+0000","last_clean":"2024-09-16T04:55:34.167452+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:34.167452+0000","last_undegraded":"2024-09-16T04:55:34.167452+0000","last_fullsized":"2024-09-16T04:55:34.167452+0000","mapping_epoch":34,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22814720,"num_objects":5570,"num_object_clones":0,"num_object_copies":11140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11516,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14482","reported_seq":14715,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.736996+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:33.736996+0000","last_peered":"2024-09-16T04:55:33.736996+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:33.736996+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22933504,"num_objects":5599,"num_object_clones":0,"num_object_copies":11198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11233,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2278}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14663","reported_seq":14798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.166367+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:34.166367+0000","last_peered":"2024-09-16T04:55:34.166367+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:34.166367+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22753280,"num_objects":5555,"num_object_clones":0,"num_object_copies":11110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11318,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2090}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14418","reported_seq":18042,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.756659+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:33.756659+0000","last_peered":"2024-09-16T04:55:33.756659+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:33.756659+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22779090,"num_objects":5563,"num_object_clones":0,"num_object_copies":11126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1610,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14390,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1690,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3953}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'14428","reported_seq":21290,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:33.753172+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:33.753172+0000","last_peered":"2024-09-16T04:55:33.753172+0000","last_clean":"2024-09-16T04:55:33.753172+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:33.753172+0000","last_undegraded":"2024-09-16T04:55:33.753172+0000","last_fullsized":"2024-09-16T04:55:33.753172+0000","mapping_epoch":46,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22732800,"num_objects":5550,"num_object_clones":0,"num_object_copies":11100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11151,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14394","reported_seq":14497,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.754573+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:33.754573+0000","last_peered":"2024-09-16T04:55:33.754573+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:33.754573+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4300","ondisk_log_start":"45'4300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22454889,"num_objects":5483,"num_object_clones":0,"num_object_copies":10966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14393,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2076}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14832","reported_seq":14938,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:33.693605+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:33.693605+0000","last_peered":"2024-09-16T04:55:33.693605+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:33.693605+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23349051,"num_objects":5703,"num_object_clones":0,"num_object_copies":11406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14829,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2189}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14638","reported_seq":21437,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:38.227674+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:38.227674+0000","last_peered":"2024-09-16T04:55:38.227674+0000","last_clean":"2024-09-16T04:55:38.227674+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:38.227674+0000","last_undegraded":"2024-09-16T04:55:38.227674+0000","last_fullsized":"2024-09-16T04:55:38.227674+0000","mapping_epoch":48,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22773760,"num_objects":5560,"num_object_clones":0,"num_object_copies":11120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11392,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14806","reported_seq":14947,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:34.220601+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:34.220601+0000","last_peered":"2024-09-16T04:55:34.220601+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:34.220601+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22979177,"num_objects":5611,"num_object_clones":0,"num_object_copies":11222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11575,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":365035767,"num_objects":89127,"num_object_clones":0,"num_object_copies":178254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197503,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8370,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":891838464,"data_stored":891764219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160625,"ondisk_log_size":160625,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663464,"kb_used_data":306904,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93708376,"statfs":{"total":96636764160,"available":95957377024,"internally_reserved":0,"allocated":314269696,"data_stored":313986370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593801,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583204,"kb_used_data":262736,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93788636,"statfs":{"total":96636764160,"available":96039563264,"internally_reserved":0,"allocated":269041664,"data_stored":268702674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":657760,"kb_used_data":305424,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93714080,"statfs":{"total":96636764160,"available":95963217920,"internally_reserved":0,"allocated":312754176,"data_stored":312474922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,0,3,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":311185408,"data_stored":311181929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":267796480,"data_stored":267733265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312856576,"data_stored":312849025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:42.650 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:43.596 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:43.598 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:43.681 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-16T04:55:42.432789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":356114698,"num_objects":86951,"num_object_clones":0,"num_object_copies":173902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32267,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86951,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199683,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8476,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160806,"ondisk_log_size":160806,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1900900,"kb_used_data":863204,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281214620,"statfs":{"total":289910292480,"available":287963770880,"internally_reserved":0,"allocated":883920896,"data_stored":883019326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1062521857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,6,5,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18984960,"num_objects":-4635,"num_object_clones":0,"num_object_copies":-9270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4635,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007541"},"pg_stats":[{"pgid":"2.f","version":"49'14414","reported_seq":14499,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:39.214660+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:39.214660+0000","last_peered":"2024-09-16T04:55:39.214660+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:39.214660+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21807104,"num_objects":5324,"num_object_clones":0,"num_object_copies":10648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14414,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1983}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14594","reported_seq":14716,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:38.768184+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:38.768184+0000","last_peered":"2024-09-16T04:55:38.768184+0000","last_clean":"2024-09-16T04:55:38.768184+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:38.768184+0000","last_undegraded":"2024-09-16T04:55:38.768184+0000","last_fullsized":"2024-09-16T04:55:38.768184+0000","mapping_epoch":35,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22118400,"num_objects":5400,"num_object_clones":0,"num_object_copies":10800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11563,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14467","reported_seq":14612,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:38.708081+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:38.708081+0000","last_peered":"2024-09-16T04:55:38.708081+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:38.708081+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4400","ondisk_log_start":"45'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22077464,"num_objects":5391,"num_object_clones":0,"num_object_copies":10782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11206,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2072}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14806","reported_seq":14882,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:38.636671+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:38.636671+0000","last_peered":"2024-09-16T04:55:38.636671+0000","last_clean":"2024-09-16T04:55:38.636671+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:38.636671+0000","last_undegraded":"2024-09-16T04:55:38.636671+0000","last_fullsized":"2024-09-16T04:55:38.636671+0000","mapping_epoch":35,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22683648,"num_objects":5538,"num_object_clones":0,"num_object_copies":11076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14806,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'14708","reported_seq":14932,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:39.222531+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:39.222531+0000","last_peered":"2024-09-16T04:55:39.222531+0000","last_clean":"2024-09-16T04:55:39.222531+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:39.222531+0000","last_undegraded":"2024-09-16T04:55:39.222531+0000","last_fullsized":"2024-09-16T04:55:39.222531+0000","mapping_epoch":34,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22265856,"num_objects":5436,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11740,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14844","reported_seq":14983,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:39.136849+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:39.136849+0000","last_peered":"2024-09-16T04:55:39.136849+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:39.136849+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22401024,"num_objects":5469,"num_object_clones":0,"num_object_copies":10938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11837,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14550","reported_seq":14704,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:39.088137+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:39.088137+0000","last_peered":"2024-09-16T04:55:39.088137+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:39.088137+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21749760,"num_objects":5310,"num_object_clones":0,"num_object_copies":10620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11205,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1911}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'14596","reported_seq":14817,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:39.140317+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:39.140317+0000","last_peered":"2024-09-16T04:55:39.140317+0000","last_clean":"2024-09-16T04:55:39.140317+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:39.140317+0000","last_undegraded":"2024-09-16T04:55:39.140317+0000","last_fullsized":"2024-09-16T04:55:39.140317+0000","mapping_epoch":34,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22200320,"num_objects":5420,"num_object_clones":0,"num_object_copies":10840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11666,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14634","reported_seq":14867,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:38.604426+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:38.604426+0000","last_peered":"2024-09-16T04:55:38.604426+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:38.604426+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22310912,"num_objects":5447,"num_object_clones":0,"num_object_copies":10894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11385,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14788","reported_seq":14923,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:39.208306+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:39.208306+0000","last_peered":"2024-09-16T04:55:39.208306+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:39.208306+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22241280,"num_objects":5430,"num_object_clones":0,"num_object_copies":10860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11443,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1965}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14559","reported_seq":18394,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:38.803421+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:38.803421+0000","last_peered":"2024-09-16T04:55:38.803421+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:38.803421+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22201554,"num_objects":5422,"num_object_clones":0,"num_object_copies":10844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1505,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14531,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1795,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3917}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'14570","reported_seq":21432,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:38.765603+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:38.765603+0000","last_peered":"2024-09-16T04:55:38.765603+0000","last_clean":"2024-09-16T04:55:38.765603+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:38.765603+0000","last_undegraded":"2024-09-16T04:55:38.765603+0000","last_fullsized":"2024-09-16T04:55:38.765603+0000","mapping_epoch":46,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22151168,"num_objects":5408,"num_object_clones":0,"num_object_copies":10816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11293,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14532","reported_seq":14635,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:38.717568+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:38.717568+0000","last_peered":"2024-09-16T04:55:38.717568+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:38.717568+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21889641,"num_objects":5345,"num_object_clones":0,"num_object_copies":10690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14531,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1938}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'14950","reported_seq":15058,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:38.739033+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:38.739033+0000","last_peered":"2024-09-16T04:55:38.739033+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:38.739033+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22865723,"num_objects":5585,"num_object_clones":0,"num_object_copies":11170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14947,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2071}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14638","reported_seq":21437,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:38.227674+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:38.227674+0000","last_peered":"2024-09-16T04:55:38.227674+0000","last_clean":"2024-09-16T04:55:38.227674+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:38.227674+0000","last_undegraded":"2024-09-16T04:55:38.227674+0000","last_fullsized":"2024-09-16T04:55:38.227674+0000","mapping_epoch":48,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22773760,"num_objects":5560,"num_object_clones":0,"num_object_copies":11120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11392,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'14953","reported_seq":15098,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:39.114780+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:39.114780+0000","last_peered":"2024-09-16T04:55:39.114780+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:39.114780+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22377065,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11722,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1940}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":356114679,"num_objects":86949,"num_object_clones":0,"num_object_copies":173898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32267,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86949,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199681,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8475,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":879751168,"data_stored":879676923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160803,"ondisk_log_size":160803,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":661800,"kb_used_data":301076,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93710040,"statfs":{"total":96636764160,"available":95959080960,"internally_reserved":0,"allocated":308301824,"data_stored":308018498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593801,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583204,"kb_used_data":262736,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93788636,"statfs":{"total":96636764160,"available":96039563264,"internally_reserved":0,"allocated":269041664,"data_stored":268702674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":655896,"kb_used_data":299392,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93715944,"statfs":{"total":96636764160,"available":95965126656,"internally_reserved":0,"allocated":306577408,"data_stored":306298154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":305123328,"data_stored":305119849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":267796480,"data_stored":267733265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":306831360,"data_stored":306823809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:43.682 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:44.488 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:44.489 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:44.518 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-16T04:55:44.435468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346464522,"num_objects":84595,"num_object_clones":0,"num_object_copies":169190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":202039,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8603,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1916964,"kb_used_data":844580,"kb_used_omap":12,"kb_used_meta":1072307,"kb_avail":281198556,"statfs":{"total":289910292480,"available":287947321344,"internally_reserved":0,"allocated":864849920,"data_stored":863948350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1098042369},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,1,5,4,9,17,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28635136,"num_objects":-6991,"num_object_clones":0,"num_object_copies":-13982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6991,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009727"},"pg_stats":[{"pgid":"2.f","version":"49'14580","reported_seq":14665,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.265368+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:44.265368+0000","last_peered":"2024-09-16T04:55:44.265368+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:44.265368+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21127168,"num_objects":5158,"num_object_clones":0,"num_object_copies":10316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14580,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1817}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14742","reported_seq":14864,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:43.832471+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:43.832471+0000","last_peered":"2024-09-16T04:55:43.832471+0000","last_clean":"2024-09-16T04:55:43.832471+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:43.832471+0000","last_undegraded":"2024-09-16T04:55:43.832471+0000","last_fullsized":"2024-09-16T04:55:43.832471+0000","mapping_epoch":35,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21512192,"num_objects":5252,"num_object_clones":0,"num_object_copies":10504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11711,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14626","reported_seq":14771,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.796875+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:43.796875+0000","last_peered":"2024-09-16T04:55:43.796875+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:43.796875+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21426200,"num_objects":5232,"num_object_clones":0,"num_object_copies":10464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11365,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1913}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14926","reported_seq":15002,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:43.757544+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:43.757544+0000","last_peered":"2024-09-16T04:55:43.757544+0000","last_clean":"2024-09-16T04:55:43.757544+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:43.757544+0000","last_undegraded":"2024-09-16T04:55:43.757544+0000","last_fullsized":"2024-09-16T04:55:43.757544+0000","mapping_epoch":35,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22192128,"num_objects":5418,"num_object_clones":0,"num_object_copies":10836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14926,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'14850","reported_seq":15074,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:44.242115+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:44.242115+0000","last_peered":"2024-09-16T04:55:44.242115+0000","last_clean":"2024-09-16T04:55:44.242115+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:44.242115+0000","last_undegraded":"2024-09-16T04:55:44.242115+0000","last_fullsized":"2024-09-16T04:55:44.242115+0000","mapping_epoch":34,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21684224,"num_objects":5294,"num_object_clones":0,"num_object_copies":10588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11882,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14980","reported_seq":15119,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.154911+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:44.154911+0000","last_peered":"2024-09-16T04:55:44.154911+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:44.154911+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21843968,"num_objects":5333,"num_object_clones":0,"num_object_copies":10666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11973,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14687","reported_seq":14840,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.207390+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:44.207390+0000","last_peered":"2024-09-16T04:55:44.207390+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:44.207390+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21188608,"num_objects":5173,"num_object_clones":0,"num_object_copies":10346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11342,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1774}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'14742","reported_seq":14963,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:44.225407+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:44.225407+0000","last_peered":"2024-09-16T04:55:44.225407+0000","last_clean":"2024-09-16T04:55:44.225407+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:44.225407+0000","last_undegraded":"2024-09-16T04:55:44.225407+0000","last_fullsized":"2024-09-16T04:55:44.225407+0000","mapping_epoch":34,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21602304,"num_objects":5274,"num_object_clones":0,"num_object_copies":10548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11812,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14774","reported_seq":15008,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.803799+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:43.803799+0000","last_peered":"2024-09-16T04:55:43.803799+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:43.803799+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21737472,"num_objects":5307,"num_object_clones":0,"num_object_copies":10614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11525,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1986}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14948","reported_seq":15077,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.176433+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:44.176433+0000","last_peered":"2024-09-16T04:55:44.176433+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:44.176433+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21585920,"num_objects":5270,"num_object_clones":0,"num_object_copies":10540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11603,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1805}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14692","reported_seq":18778,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.815588+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:43.815588+0000","last_peered":"2024-09-16T04:55:43.815588+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:43.815588+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21656786,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1378,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14664,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1922,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3911}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'14714","reported_seq":21576,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:43.809946+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:43.809946+0000","last_peered":"2024-09-16T04:55:43.809946+0000","last_clean":"2024-09-16T04:55:43.809946+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:43.809946+0000","last_undegraded":"2024-09-16T04:55:43.809946+0000","last_fullsized":"2024-09-16T04:55:43.809946+0000","mapping_epoch":46,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561344,"num_objects":5264,"num_object_clones":0,"num_object_copies":10528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11437,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14701","reported_seq":14804,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.797470+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:43.797470+0000","last_peered":"2024-09-16T04:55:43.797470+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:43.797470+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21197417,"num_objects":5176,"num_object_clones":0,"num_object_copies":10352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14700,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1769}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15104","reported_seq":15212,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.630919+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:43.630919+0000","last_peered":"2024-09-16T04:55:43.630919+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:43.630919+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22234939,"num_objects":5431,"num_object_clones":0,"num_object_copies":10862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15101,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1917}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14786","reported_seq":21585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:43.199144+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:43.199144+0000","last_peered":"2024-09-16T04:55:43.199144+0000","last_clean":"2024-09-16T04:55:43.199144+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:43.199144+0000","last_undegraded":"2024-09-16T04:55:43.199144+0000","last_fullsized":"2024-09-16T04:55:43.199144+0000","mapping_epoch":48,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22167552,"num_objects":5412,"num_object_clones":0,"num_object_copies":10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11540,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15107","reported_seq":15251,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.236021+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:44.236021+0000","last_peered":"2024-09-16T04:55:44.236021+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:44.236021+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21746281,"num_objects":5310,"num_object_clones":0,"num_object_copies":10620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11876,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1786}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":346464503,"num_objects":84593,"num_object_clones":0,"num_object_copies":169186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84593,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":202037,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8602,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":860557312,"data_stored":860483067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160759,"ondisk_log_size":160759,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660188,"kb_used_data":295300,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93711652,"statfs":{"total":96636764160,"available":95960731648,"internally_reserved":0,"allocated":302387200,"data_stored":302103874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,1,0,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593802,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602400,"kb_used_data":255560,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93769440,"statfs":{"total":96636764160,"available":96019906560,"internally_reserved":0,"allocated":261693440,"data_stored":261354450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,5,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":654376,"kb_used_data":293720,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93717464,"statfs":{"total":96636764160,"available":95966683136,"internally_reserved":0,"allocated":300769280,"data_stored":300490026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299155456,"data_stored":299151977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260497408,"data_stored":260434193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":300904448,"data_stored":300896897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:44.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:55:44.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'14580", 'reported_seq': 14665, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:44.265368+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:55:44.265368+0000', 'last_peered': '2024-09-16T04:55:44.265368+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:55:44.265368+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'4500", 'ondisk_log_start': "45'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21127168, 'num_objects': 5158, 'num_object_clones': 0, 'num_object_copies': 10316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14580, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1817}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:44.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:55:44.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'14626", 'reported_seq': 14771, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:43.796875+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:55:43.796875+0000', 'last_peered': '2024-09-16T04:55:43.796875+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:55:43.796875+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'4600", 'ondisk_log_start': "45'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21426200, 'num_objects': 5232, 'num_object_clones': 0, 'num_object_copies': 10464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5232, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11365, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1913}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:44.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:55:44.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'14980", 'reported_seq': 15119, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:44.154911+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:55:44.154911+0000', 'last_peered': '2024-09-16T04:55:44.154911+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:55:44.154911+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'4900", 'ondisk_log_start': "45'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21843968, 'num_objects': 5333, 'num_object_clones': 0, 'num_object_copies': 10666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5333, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11973, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:44.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:55:44.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'14687", 'reported_seq': 14840, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:44.207390+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:55:44.207390+0000', 'last_peered': '2024-09-16T04:55:44.207390+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:55:44.207390+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'4600", 'ondisk_log_start': "45'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21188608, 'num_objects': 5173, 'num_object_clones': 0, 'num_object_copies': 10346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5173, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11342, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1774}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:44.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:55:44.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'14774", 'reported_seq': 15008, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:43.803799+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:55:43.803799+0000', 'last_peered': '2024-09-16T04:55:43.803799+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:55:43.803799+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'4700", 'ondisk_log_start': "45'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21737472, 'num_objects': 5307, 'num_object_clones': 0, 'num_object_copies': 10614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5307, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11525, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:44.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:55:44.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'14948", 'reported_seq': 15077, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:44.176433+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:55:44.176433+0000', 'last_peered': '2024-09-16T04:55:44.176433+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:55:44.176433+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'4900", 'ondisk_log_start': "45'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21585920, 'num_objects': 5270, 'num_object_clones': 0, 'num_object_copies': 10540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5270, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11603, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1805}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:44.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:55:44.521 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'14692", 'reported_seq': 18778, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:43.815588+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:55:43.815588+0000', 'last_peered': '2024-09-16T04:55:43.815588+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:55:43.815588+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'4600", 'ondisk_log_start': "45'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21656786, 'num_objects': 5289, 'num_object_clones': 0, 'num_object_copies': 10578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1378, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5289, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14664, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1922, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3911}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:44.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:55:44.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'14701", 'reported_seq': 14804, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:43.797470+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:55:43.797470+0000', 'last_peered': '2024-09-16T04:55:43.797470+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:55:43.797470+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'4700", 'ondisk_log_start': "45'4700", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21197417, 'num_objects': 5176, 'num_object_clones': 0, 'num_object_copies': 10352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5176, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14700, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1769}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:44.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:55:44.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'15104", 'reported_seq': 15212, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:43.630919+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:55:43.630919+0000', 'last_peered': '2024-09-16T04:55:43.630919+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:55:43.630919+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'5100", 'ondisk_log_start': "45'5100", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22234939, 'num_objects': 5431, 'num_object_clones': 0, 'num_object_copies': 10862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5431, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15101, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:44.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:55:44.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'15107", 'reported_seq': 15251, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:44.236021+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:55:44.236021+0000', 'last_peered': '2024-09-16T04:55:44.236021+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:55:44.236021+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'5100", 'ondisk_log_start': "45'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21746281, 'num_objects': 5310, 'num_object_clones': 0, 'num_object_copies': 10620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5310, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11876, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1786}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:44.523 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:55:45.520 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:45.520 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32140/169190 objects degraded (18.996%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":493,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":84595,"data_bytes":346464522,"bytes_used":1962971136,"bytes_avail":287947321344,"bytes_total":289910292480,"degraded_objects":32140,"degraded_total":169190,"degraded_ratio":0.18996394585968437,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":331,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":582},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:55:45.593 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:55:45.593 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:46.435 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:46.530 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-16T04:55:44.435468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346464522,"num_objects":84595,"num_object_clones":0,"num_object_copies":169190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":202039,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8603,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160762,"ondisk_log_size":160762,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1916964,"kb_used_data":844580,"kb_used_omap":12,"kb_used_meta":1072307,"kb_avail":281198556,"statfs":{"total":289910292480,"available":287947321344,"internally_reserved":0,"allocated":864849920,"data_stored":863948350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1098042369},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,1,5,4,9,17,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28635136,"num_objects":-6991,"num_object_clones":0,"num_object_copies":-13982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6991,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009727"},"pg_stats":[{"pgid":"2.f","version":"49'14580","reported_seq":14665,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.265368+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:44.265368+0000","last_peered":"2024-09-16T04:55:44.265368+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:44.265368+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21127168,"num_objects":5158,"num_object_clones":0,"num_object_copies":10316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14580,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1817}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14742","reported_seq":14864,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:43.832471+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:43.832471+0000","last_peered":"2024-09-16T04:55:43.832471+0000","last_clean":"2024-09-16T04:55:43.832471+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:43.832471+0000","last_undegraded":"2024-09-16T04:55:43.832471+0000","last_fullsized":"2024-09-16T04:55:43.832471+0000","mapping_epoch":35,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21512192,"num_objects":5252,"num_object_clones":0,"num_object_copies":10504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11711,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14626","reported_seq":14771,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.796875+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:43.796875+0000","last_peered":"2024-09-16T04:55:43.796875+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:43.796875+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21426200,"num_objects":5232,"num_object_clones":0,"num_object_copies":10464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11365,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1913}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14926","reported_seq":15002,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:43.757544+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:43.757544+0000","last_peered":"2024-09-16T04:55:43.757544+0000","last_clean":"2024-09-16T04:55:43.757544+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:43.757544+0000","last_undegraded":"2024-09-16T04:55:43.757544+0000","last_fullsized":"2024-09-16T04:55:43.757544+0000","mapping_epoch":35,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22192128,"num_objects":5418,"num_object_clones":0,"num_object_copies":10836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14926,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'14850","reported_seq":15074,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:44.242115+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:44.242115+0000","last_peered":"2024-09-16T04:55:44.242115+0000","last_clean":"2024-09-16T04:55:44.242115+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:44.242115+0000","last_undegraded":"2024-09-16T04:55:44.242115+0000","last_fullsized":"2024-09-16T04:55:44.242115+0000","mapping_epoch":34,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21684224,"num_objects":5294,"num_object_clones":0,"num_object_copies":10588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11882,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14980","reported_seq":15119,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.154911+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:44.154911+0000","last_peered":"2024-09-16T04:55:44.154911+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:44.154911+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21843968,"num_objects":5333,"num_object_clones":0,"num_object_copies":10666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11973,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14687","reported_seq":14840,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.207390+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:44.207390+0000","last_peered":"2024-09-16T04:55:44.207390+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:44.207390+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21188608,"num_objects":5173,"num_object_clones":0,"num_object_copies":10346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11342,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1774}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'14742","reported_seq":14963,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:44.225407+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:44.225407+0000","last_peered":"2024-09-16T04:55:44.225407+0000","last_clean":"2024-09-16T04:55:44.225407+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:44.225407+0000","last_undegraded":"2024-09-16T04:55:44.225407+0000","last_fullsized":"2024-09-16T04:55:44.225407+0000","mapping_epoch":34,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21602304,"num_objects":5274,"num_object_clones":0,"num_object_copies":10548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11812,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14774","reported_seq":15008,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.803799+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:43.803799+0000","last_peered":"2024-09-16T04:55:43.803799+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:43.803799+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21737472,"num_objects":5307,"num_object_clones":0,"num_object_copies":10614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11525,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1986}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14948","reported_seq":15077,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.176433+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:44.176433+0000","last_peered":"2024-09-16T04:55:44.176433+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:44.176433+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21585920,"num_objects":5270,"num_object_clones":0,"num_object_copies":10540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11603,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1805}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14692","reported_seq":18778,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.815588+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:43.815588+0000","last_peered":"2024-09-16T04:55:43.815588+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:43.815588+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21656786,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1378,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14664,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1922,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3911}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'14714","reported_seq":21576,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:43.809946+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:43.809946+0000","last_peered":"2024-09-16T04:55:43.809946+0000","last_clean":"2024-09-16T04:55:43.809946+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:43.809946+0000","last_undegraded":"2024-09-16T04:55:43.809946+0000","last_fullsized":"2024-09-16T04:55:43.809946+0000","mapping_epoch":46,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561344,"num_objects":5264,"num_object_clones":0,"num_object_copies":10528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11437,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14701","reported_seq":14804,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.797470+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:43.797470+0000","last_peered":"2024-09-16T04:55:43.797470+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:43.797470+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21197417,"num_objects":5176,"num_object_clones":0,"num_object_copies":10352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14700,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1769}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15104","reported_seq":15212,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.630919+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:43.630919+0000","last_peered":"2024-09-16T04:55:43.630919+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:43.630919+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22234939,"num_objects":5431,"num_object_clones":0,"num_object_copies":10862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15101,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1917}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14786","reported_seq":21585,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:43.199144+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:43.199144+0000","last_peered":"2024-09-16T04:55:43.199144+0000","last_clean":"2024-09-16T04:55:43.199144+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:43.199144+0000","last_undegraded":"2024-09-16T04:55:43.199144+0000","last_fullsized":"2024-09-16T04:55:43.199144+0000","mapping_epoch":48,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22167552,"num_objects":5412,"num_object_clones":0,"num_object_copies":10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11540,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15107","reported_seq":15251,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.236021+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:44.236021+0000","last_peered":"2024-09-16T04:55:44.236021+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:44.236021+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21746281,"num_objects":5310,"num_object_clones":0,"num_object_copies":10620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11876,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1786}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":346464503,"num_objects":84593,"num_object_clones":0,"num_object_copies":169186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84593,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":202037,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8602,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":860557312,"data_stored":860483067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160759,"ondisk_log_size":160759,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660188,"kb_used_data":295300,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93711652,"statfs":{"total":96636764160,"available":95960731648,"internally_reserved":0,"allocated":302387200,"data_stored":302103874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,1,0,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593802,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602400,"kb_used_data":255560,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93769440,"statfs":{"total":96636764160,"available":96019906560,"internally_reserved":0,"allocated":261693440,"data_stored":261354450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,5,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":654376,"kb_used_data":293720,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93717464,"statfs":{"total":96636764160,"available":95966683136,"internally_reserved":0,"allocated":300769280,"data_stored":300490026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299155456,"data_stored":299151977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260497408,"data_stored":260434193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":300904448,"data_stored":300896897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:49.532 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:50.271 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:50.272 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:50.322 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-16T04:55:48.436790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":345903370,"num_objects":84458,"num_object_clones":0,"num_object_copies":168916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":202176,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8603,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1901800,"kb_used_data":837740,"kb_used_omap":12,"kb_used_meta":1063987,"kb_avail":281213720,"statfs":{"total":289910292480,"available":287962849280,"internally_reserved":0,"allocated":857845760,"data_stored":856944190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1089522689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,1,3,4,7,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19779584,"num_objects":-4829,"num_object_clones":0,"num_object_copies":-9658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4829,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010136"},"pg_stats":[{"pgid":"2.f","version":"49'14580","reported_seq":14665,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.265368+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:44.265368+0000","last_peered":"2024-09-16T04:55:44.265368+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:44.265368+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4500","ondisk_log_start":"45'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21127168,"num_objects":5158,"num_object_clones":0,"num_object_copies":10316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14580,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1817}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14742","reported_seq":14864,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:43.832471+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:43.832471+0000","last_peered":"2024-09-16T04:55:43.832471+0000","last_clean":"2024-09-16T04:55:43.832471+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:43.832471+0000","last_undegraded":"2024-09-16T04:55:43.832471+0000","last_fullsized":"2024-09-16T04:55:43.832471+0000","mapping_epoch":35,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21512192,"num_objects":5252,"num_object_clones":0,"num_object_copies":10504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11711,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14626","reported_seq":14771,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.796875+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:43.796875+0000","last_peered":"2024-09-16T04:55:43.796875+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:43.796875+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21426200,"num_objects":5232,"num_object_clones":0,"num_object_copies":10464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5232,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11365,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1913}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'14926","reported_seq":15002,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:43.757544+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:43.757544+0000","last_peered":"2024-09-16T04:55:43.757544+0000","last_clean":"2024-09-16T04:55:43.757544+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:43.757544+0000","last_undegraded":"2024-09-16T04:55:43.757544+0000","last_fullsized":"2024-09-16T04:55:43.757544+0000","mapping_epoch":35,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22192128,"num_objects":5418,"num_object_clones":0,"num_object_copies":10836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14926,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'14850","reported_seq":15074,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:44.242115+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:44.242115+0000","last_peered":"2024-09-16T04:55:44.242115+0000","last_clean":"2024-09-16T04:55:44.242115+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:44.242115+0000","last_undegraded":"2024-09-16T04:55:44.242115+0000","last_fullsized":"2024-09-16T04:55:44.242115+0000","mapping_epoch":34,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21684224,"num_objects":5294,"num_object_clones":0,"num_object_copies":10588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11882,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'14980","reported_seq":15119,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.154911+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:44.154911+0000","last_peered":"2024-09-16T04:55:44.154911+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:44.154911+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21843968,"num_objects":5333,"num_object_clones":0,"num_object_copies":10666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11973,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14687","reported_seq":14840,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.207390+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:44.207390+0000","last_peered":"2024-09-16T04:55:44.207390+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:44.207390+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21188608,"num_objects":5173,"num_object_clones":0,"num_object_copies":10346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11342,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1774}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'14742","reported_seq":14963,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:44.225407+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:44.225407+0000","last_peered":"2024-09-16T04:55:44.225407+0000","last_clean":"2024-09-16T04:55:44.225407+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:44.225407+0000","last_undegraded":"2024-09-16T04:55:44.225407+0000","last_fullsized":"2024-09-16T04:55:44.225407+0000","mapping_epoch":34,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21602304,"num_objects":5274,"num_object_clones":0,"num_object_copies":10548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11812,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14774","reported_seq":15008,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.803799+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:43.803799+0000","last_peered":"2024-09-16T04:55:43.803799+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:43.803799+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21737472,"num_objects":5307,"num_object_clones":0,"num_object_copies":10614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11525,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1986}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'14948","reported_seq":15077,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.176433+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:44.176433+0000","last_peered":"2024-09-16T04:55:44.176433+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:44.176433+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21585920,"num_objects":5270,"num_object_clones":0,"num_object_copies":10540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11603,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1805}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14692","reported_seq":18778,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.815588+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:43.815588+0000","last_peered":"2024-09-16T04:55:43.815588+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:43.815588+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4600","ondisk_log_start":"45'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21656786,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1378,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14664,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1922,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3911}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'14714","reported_seq":21576,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:43.809946+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:43.809946+0000","last_peered":"2024-09-16T04:55:43.809946+0000","last_clean":"2024-09-16T04:55:43.809946+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:43.809946+0000","last_undegraded":"2024-09-16T04:55:43.809946+0000","last_fullsized":"2024-09-16T04:55:43.809946+0000","mapping_epoch":46,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561344,"num_objects":5264,"num_object_clones":0,"num_object_copies":10528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11437,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14701","reported_seq":14804,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.797470+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:43.797470+0000","last_peered":"2024-09-16T04:55:43.797470+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:43.797470+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21197417,"num_objects":5176,"num_object_clones":0,"num_object_copies":10352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14700,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1769}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15104","reported_seq":15212,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:43.630919+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:43.630919+0000","last_peered":"2024-09-16T04:55:43.630919+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:43.630919+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22234939,"num_objects":5431,"num_object_clones":0,"num_object_copies":10862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15101,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1917}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14923","reported_seq":21722,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:48.212931+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:48.212931+0000","last_peered":"2024-09-16T04:55:48.212931+0000","last_clean":"2024-09-16T04:55:48.212931+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:48.212931+0000","last_undegraded":"2024-09-16T04:55:48.212931+0000","last_fullsized":"2024-09-16T04:55:48.212931+0000","mapping_epoch":48,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21606400,"num_objects":5275,"num_object_clones":0,"num_object_copies":10550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11677,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15107","reported_seq":15251,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:44.236021+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:44.236021+0000","last_peered":"2024-09-16T04:55:44.236021+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:44.236021+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21746281,"num_objects":5310,"num_object_clones":0,"num_object_copies":10620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11876,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1786}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":345903351,"num_objects":84456,"num_object_clones":0,"num_object_copies":168912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84456,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":202174,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8602,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":853446656,"data_stored":853372411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160696,"ondisk_log_size":160696,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":660188,"kb_used_data":295300,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93711652,"statfs":{"total":96636764160,"available":95960731648,"internally_reserved":0,"allocated":302387200,"data_stored":302103874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,1,0,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593803,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587236,"kb_used_data":248720,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93784604,"statfs":{"total":96636764160,"available":96035434496,"internally_reserved":0,"allocated":254689280,"data_stored":254350290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":654376,"kb_used_data":293720,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93717464,"statfs":{"total":96636764160,"available":95966683136,"internally_reserved":0,"allocated":300769280,"data_stored":300490026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299155456,"data_stored":299151977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253386752,"data_stored":253323537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":300904448,"data_stored":300896897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:50.323 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:51.251 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:51.254 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:51.308 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-16T04:55:50.437151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":336978186,"num_objects":82279,"num_object_clones":0,"num_object_copies":164558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82279,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204355,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8717,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160678,"ondisk_log_size":160678,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1886136,"kb_used_data":826240,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281229384,"statfs":{"total":289910292480,"available":287978889216,"internally_reserved":0,"allocated":846069760,"data_stored":845168190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1085262849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,8,7,9,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28057600,"num_objects":-6850,"num_object_clones":0,"num_object_copies":-13700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-346,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6850,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010084"},"pg_stats":[{"pgid":"2.f","version":"49'14722","reported_seq":14807,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:49.254548+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:49.254548+0000","last_peered":"2024-09-16T04:55:49.254548+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:49.254548+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20545536,"num_objects":5016,"num_object_clones":0,"num_object_copies":10032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14722,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14892","reported_seq":15014,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:48.872494+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:48.872494+0000","last_peered":"2024-09-16T04:55:48.872494+0000","last_clean":"2024-09-16T04:55:48.872494+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:48.872494+0000","last_undegraded":"2024-09-16T04:55:48.872494+0000","last_fullsized":"2024-09-16T04:55:48.872494+0000","mapping_epoch":35,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20897792,"num_objects":5102,"num_object_clones":0,"num_object_copies":10204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11861,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14771","reported_seq":14916,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:48.820779+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:48.820779+0000","last_peered":"2024-09-16T04:55:48.820779+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:48.820779+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20832280,"num_objects":5087,"num_object_clones":0,"num_object_copies":10174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11510,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1768}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15064","reported_seq":15140,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:48.723521+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:48.723521+0000","last_peered":"2024-09-16T04:55:48.723521+0000","last_clean":"2024-09-16T04:55:48.723521+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:48.723521+0000","last_undegraded":"2024-09-16T04:55:48.723521+0000","last_fullsized":"2024-09-16T04:55:48.723521+0000","mapping_epoch":35,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21626880,"num_objects":5280,"num_object_clones":0,"num_object_copies":10560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15064,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15015","reported_seq":15239,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:49.209186+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:49.209186+0000","last_peered":"2024-09-16T04:55:49.209186+0000","last_clean":"2024-09-16T04:55:49.209186+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:49.209186+0000","last_undegraded":"2024-09-16T04:55:49.209186+0000","last_fullsized":"2024-09-16T04:55:49.209186+0000","mapping_epoch":34,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21008384,"num_objects":5129,"num_object_clones":0,"num_object_copies":10258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12047,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15131","reported_seq":15270,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:49.239066+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:49.239066+0000","last_peered":"2024-09-16T04:55:49.239066+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:49.239066+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21225472,"num_objects":5182,"num_object_clones":0,"num_object_copies":10364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12124,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1726}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14836","reported_seq":14989,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:49.204135+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:49.204135+0000","last_peered":"2024-09-16T04:55:49.204135+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:49.204135+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":5024,"num_object_clones":0,"num_object_copies":10048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11491,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1625}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'14882","reported_seq":15103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:49.272152+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:49.272152+0000","last_peered":"2024-09-16T04:55:49.272152+0000","last_clean":"2024-09-16T04:55:49.272152+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:49.272152+0000","last_undegraded":"2024-09-16T04:55:49.272152+0000","last_fullsized":"2024-09-16T04:55:49.272152+0000","mapping_epoch":34,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21028864,"num_objects":5134,"num_object_clones":0,"num_object_copies":10268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11952,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14904","reported_seq":15138,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:48.815558+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:48.815558+0000","last_peered":"2024-09-16T04:55:48.815558+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:48.815558+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21204992,"num_objects":5177,"num_object_clones":0,"num_object_copies":10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11655,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1856}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15106","reported_seq":15241,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:49.265345+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:49.265345+0000","last_peered":"2024-09-16T04:55:49.265345+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:49.265345+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20938752,"num_objects":5112,"num_object_clones":0,"num_object_copies":10224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11761,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1647}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14846","reported_seq":19161,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:48.834716+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:48.834716+0000","last_peered":"2024-09-16T04:55:48.834716+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:48.834716+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21026002,"num_objects":5135,"num_object_clones":0,"num_object_copies":10270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14818,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2036,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3871}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'14867","reported_seq":21729,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:48.852255+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:48.852255+0000","last_peered":"2024-09-16T04:55:48.852255+0000","last_clean":"2024-09-16T04:55:48.852255+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:48.852255+0000","last_undegraded":"2024-09-16T04:55:48.852255+0000","last_fullsized":"2024-09-16T04:55:48.852255+0000","mapping_epoch":46,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20934656,"num_objects":5111,"num_object_clones":0,"num_object_copies":10222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11590,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14827","reported_seq":14930,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:48.780111+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:48.780111+0000","last_peered":"2024-09-16T04:55:48.780111+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:48.780111+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20681321,"num_objects":5050,"num_object_clones":0,"num_object_copies":10100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14826,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1643}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15249","reported_seq":15357,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:48.811569+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:48.811569+0000","last_peered":"2024-09-16T04:55:48.811569+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:48.811569+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21641019,"num_objects":5286,"num_object_clones":0,"num_object_copies":10572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15246,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1772}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14923","reported_seq":21722,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:48.212931+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:48.212931+0000","last_peered":"2024-09-16T04:55:48.212931+0000","last_clean":"2024-09-16T04:55:48.212931+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:48.212931+0000","last_undegraded":"2024-09-16T04:55:48.212931+0000","last_fullsized":"2024-09-16T04:55:48.212931+0000","mapping_epoch":48,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21606400,"num_objects":5275,"num_object_clones":0,"num_object_copies":10550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11677,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15240","reported_seq":15385,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:49.136121+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:49.136121+0000","last_peered":"2024-09-16T04:55:49.136121+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:49.136121+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21201513,"num_objects":5177,"num_object_clones":0,"num_object_copies":10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12009,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1653}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":336978167,"num_objects":82277,"num_object_clones":0,"num_object_copies":164554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82277,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204353,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8716,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":841453568,"data_stored":841379323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160675,"ondisk_log_size":160675,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":641988,"kb_used_data":289588,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93729852,"statfs":{"total":96636764160,"available":95979368448,"internally_reserved":0,"allocated":296538112,"data_stored":296254786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,5,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593803,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587236,"kb_used_data":248720,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93784604,"statfs":{"total":96636764160,"available":96035434496,"internally_reserved":0,"allocated":254689280,"data_stored":254350290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656912,"kb_used_data":287932,"kb_used_omap":4,"kb_used_meta":368955,"kb_avail":93714928,"statfs":{"total":96636764160,"available":95964086272,"internally_reserved":0,"allocated":294842368,"data_stored":294563114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377810602},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":293126144,"data_stored":293122665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253386752,"data_stored":253323537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":294940672,"data_stored":294933121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:51.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:55:51.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'14722", 'reported_seq': 14807, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:49.254548+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:55:49.254548+0000', 'last_peered': '2024-09-16T04:55:49.254548+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:55:49.254548+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'4700", 'ondisk_log_start': "45'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20545536, 'num_objects': 5016, 'num_object_clones': 0, 'num_object_copies': 10032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14722, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1675}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:51.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:55:51.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'14771", 'reported_seq': 14916, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:48.820779+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:55:48.820779+0000', 'last_peered': '2024-09-16T04:55:48.820779+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:55:48.820779+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'4700", 'ondisk_log_start': "45'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20832280, 'num_objects': 5087, 'num_object_clones': 0, 'num_object_copies': 10174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5087, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11510, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1768}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:51.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:55:51.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'15131", 'reported_seq': 15270, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:49.239066+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:55:49.239066+0000', 'last_peered': '2024-09-16T04:55:49.239066+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:55:49.239066+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'5100", 'ondisk_log_start': "45'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21225472, 'num_objects': 5182, 'num_object_clones': 0, 'num_object_copies': 10364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5182, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12124, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1726}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:51.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:55:51.310 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'14836", 'reported_seq': 14989, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:49.204135+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:55:49.204135+0000', 'last_peered': '2024-09-16T04:55:49.204135+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:55:49.204135+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'4800", 'ondisk_log_start': "45'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20578304, 'num_objects': 5024, 'num_object_clones': 0, 'num_object_copies': 10048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11491, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1625}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:51.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:55:51.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'14904", 'reported_seq': 15138, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:48.815558+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:55:48.815558+0000', 'last_peered': '2024-09-16T04:55:48.815558+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:55:48.815558+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'4900", 'ondisk_log_start': "45'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21204992, 'num_objects': 5177, 'num_object_clones': 0, 'num_object_copies': 10354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11655, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1856}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:51.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:55:51.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'15106", 'reported_seq': 15241, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:49.265345+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:55:49.265345+0000', 'last_peered': '2024-09-16T04:55:49.265345+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:55:49.265345+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'5100", 'ondisk_log_start': "45'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20938752, 'num_objects': 5112, 'num_object_clones': 0, 'num_object_copies': 10224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5112, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11761, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1647}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:51.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:55:51.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'14846", 'reported_seq': 19161, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:48.834716+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:55:48.834716+0000', 'last_peered': '2024-09-16T04:55:48.834716+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:55:48.834716+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'4800", 'ondisk_log_start': "45'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21026002, 'num_objects': 5135, 'num_object_clones': 0, 'num_object_copies': 10270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1264, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5135, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14818, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2036, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3871}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:51.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:55:51.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'14827", 'reported_seq': 14930, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:48.780111+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:55:48.780111+0000', 'last_peered': '2024-09-16T04:55:48.780111+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:55:48.780111+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'4800", 'ondisk_log_start': "45'4800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20681321, 'num_objects': 5050, 'num_object_clones': 0, 'num_object_copies': 10100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5050, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14826, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1643}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:51.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:55:51.311 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'15249", 'reported_seq': 15357, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:48.811569+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:55:48.811569+0000', 'last_peered': '2024-09-16T04:55:48.811569+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:55:48.811569+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'5200", 'ondisk_log_start': "45'5200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21641019, 'num_objects': 5286, 'num_object_clones': 0, 'num_object_copies': 10572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5286, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15246, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1772}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:51.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:55:51.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'15240", 'reported_seq': 15385, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:49.136121+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:55:49.136121+0000', 'last_peered': '2024-09-16T04:55:49.136121+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:55:49.136121+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'5200", 'ondisk_log_start': "45'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21201513, 'num_objects': 5177, 'num_object_clones': 0, 'num_object_copies': 10354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12009, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1653}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:51.312 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:55:52.341 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:52.341 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32026/164558 objects degraded (19.462%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":500,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":82279,"data_bytes":336978186,"bytes_used":1931403264,"bytes_avail":287978889216,"bytes_total":289910292480,"degraded_objects":32026,"degraded_total":164558,"degraded_ratio":0.19461831086911605,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":346,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":570},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:55:52.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:55:52.395 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:53.169 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:53.171 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:53.253 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-16T04:55:52.439830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":336978186,"num_objects":82279,"num_object_clones":0,"num_object_copies":164558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82279,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204355,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8717,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160678,"ondisk_log_size":160678,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1886136,"kb_used_data":826240,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281229384,"statfs":{"total":289910292480,"available":287978889216,"internally_reserved":0,"allocated":846069760,"data_stored":845168190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1085262849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,8,7,9,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19136512,"num_objects":-4672,"num_object_clones":0,"num_object_copies":-9344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-241,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4672,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012334"},"pg_stats":[{"pgid":"2.f","version":"49'14722","reported_seq":14807,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:49.254548+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:49.254548+0000","last_peered":"2024-09-16T04:55:49.254548+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:49.254548+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20545536,"num_objects":5016,"num_object_clones":0,"num_object_copies":10032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14722,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'14892","reported_seq":15014,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:48.872494+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:48.872494+0000","last_peered":"2024-09-16T04:55:48.872494+0000","last_clean":"2024-09-16T04:55:48.872494+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:48.872494+0000","last_undegraded":"2024-09-16T04:55:48.872494+0000","last_fullsized":"2024-09-16T04:55:48.872494+0000","mapping_epoch":35,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20897792,"num_objects":5102,"num_object_clones":0,"num_object_copies":10204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11861,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14771","reported_seq":14916,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:48.820779+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:48.820779+0000","last_peered":"2024-09-16T04:55:48.820779+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:48.820779+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4700","ondisk_log_start":"45'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20832280,"num_objects":5087,"num_object_clones":0,"num_object_copies":10174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11510,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1768}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15064","reported_seq":15140,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:48.723521+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:48.723521+0000","last_peered":"2024-09-16T04:55:48.723521+0000","last_clean":"2024-09-16T04:55:48.723521+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:48.723521+0000","last_undegraded":"2024-09-16T04:55:48.723521+0000","last_fullsized":"2024-09-16T04:55:48.723521+0000","mapping_epoch":35,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21626880,"num_objects":5280,"num_object_clones":0,"num_object_copies":10560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15064,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15015","reported_seq":15239,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:49.209186+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:49.209186+0000","last_peered":"2024-09-16T04:55:49.209186+0000","last_clean":"2024-09-16T04:55:49.209186+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:49.209186+0000","last_undegraded":"2024-09-16T04:55:49.209186+0000","last_fullsized":"2024-09-16T04:55:49.209186+0000","mapping_epoch":34,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21008384,"num_objects":5129,"num_object_clones":0,"num_object_copies":10258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12047,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15131","reported_seq":15270,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:49.239066+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:49.239066+0000","last_peered":"2024-09-16T04:55:49.239066+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:49.239066+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21225472,"num_objects":5182,"num_object_clones":0,"num_object_copies":10364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12124,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1726}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14836","reported_seq":14989,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:49.204135+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:49.204135+0000","last_peered":"2024-09-16T04:55:49.204135+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:49.204135+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":5024,"num_object_clones":0,"num_object_copies":10048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11491,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1625}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'14882","reported_seq":15103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:49.272152+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:49.272152+0000","last_peered":"2024-09-16T04:55:49.272152+0000","last_clean":"2024-09-16T04:55:49.272152+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:49.272152+0000","last_undegraded":"2024-09-16T04:55:49.272152+0000","last_fullsized":"2024-09-16T04:55:49.272152+0000","mapping_epoch":34,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21028864,"num_objects":5134,"num_object_clones":0,"num_object_copies":10268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11952,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'14904","reported_seq":15138,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:48.815558+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:48.815558+0000","last_peered":"2024-09-16T04:55:48.815558+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:48.815558+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21204992,"num_objects":5177,"num_object_clones":0,"num_object_copies":10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11655,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1856}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15106","reported_seq":15241,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:49.265345+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:49.265345+0000","last_peered":"2024-09-16T04:55:49.265345+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:49.265345+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20938752,"num_objects":5112,"num_object_clones":0,"num_object_copies":10224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11761,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1647}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14846","reported_seq":19161,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:48.834716+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:48.834716+0000","last_peered":"2024-09-16T04:55:48.834716+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:48.834716+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21026002,"num_objects":5135,"num_object_clones":0,"num_object_copies":10270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14818,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2036,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3871}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567433+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567433+0000","last_peered":"2024-09-16T04:54:05.567433+0000","last_clean":"2024-09-16T04:54:05.567433+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567433+0000","last_undegraded":"2024-09-16T04:54:05.567433+0000","last_fullsized":"2024-09-16T04:54:05.567433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'14867","reported_seq":21729,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:48.852255+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:48.852255+0000","last_peered":"2024-09-16T04:55:48.852255+0000","last_clean":"2024-09-16T04:55:48.852255+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:48.852255+0000","last_undegraded":"2024-09-16T04:55:48.852255+0000","last_fullsized":"2024-09-16T04:55:48.852255+0000","mapping_epoch":46,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20934656,"num_objects":5111,"num_object_clones":0,"num_object_copies":10222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11590,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14827","reported_seq":14930,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:48.780111+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:48.780111+0000","last_peered":"2024-09-16T04:55:48.780111+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:48.780111+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20681321,"num_objects":5050,"num_object_clones":0,"num_object_copies":10100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14826,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1643}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567546+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567546+0000","last_peered":"2024-09-16T04:54:05.567546+0000","last_clean":"2024-09-16T04:54:05.567546+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567546+0000","last_undegraded":"2024-09-16T04:54:05.567546+0000","last_fullsized":"2024-09-16T04:54:05.567546+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15249","reported_seq":15357,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:48.811569+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:48.811569+0000","last_peered":"2024-09-16T04:55:48.811569+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:48.811569+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21641019,"num_objects":5286,"num_object_clones":0,"num_object_copies":10572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15246,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1772}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'14923","reported_seq":21722,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:48.212931+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:48.212931+0000","last_peered":"2024-09-16T04:55:48.212931+0000","last_clean":"2024-09-16T04:55:48.212931+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:48.212931+0000","last_undegraded":"2024-09-16T04:55:48.212931+0000","last_fullsized":"2024-09-16T04:55:48.212931+0000","mapping_epoch":48,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21606400,"num_objects":5275,"num_object_clones":0,"num_object_copies":10550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11677,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15240","reported_seq":15385,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:49.136121+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:49.136121+0000","last_peered":"2024-09-16T04:55:49.136121+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:49.136121+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21201513,"num_objects":5177,"num_object_clones":0,"num_object_copies":10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12009,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1653}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":336978167,"num_objects":82277,"num_object_clones":0,"num_object_copies":164554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82277,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204353,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8716,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":841453568,"data_stored":841379323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160675,"ondisk_log_size":160675,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":641988,"kb_used_data":289588,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93729852,"statfs":{"total":96636764160,"available":95979368448,"internally_reserved":0,"allocated":296538112,"data_stored":296254786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,5,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593803,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587236,"kb_used_data":248720,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93784604,"statfs":{"total":96636764160,"available":96035434496,"internally_reserved":0,"allocated":254689280,"data_stored":254350290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656912,"kb_used_data":287932,"kb_used_omap":4,"kb_used_meta":368955,"kb_avail":93714928,"statfs":{"total":96636764160,"available":95964086272,"internally_reserved":0,"allocated":294842368,"data_stored":294563114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":377810602},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":293126144,"data_stored":293122665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253386752,"data_stored":253323537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":294940672,"data_stored":294933121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:56.255 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:56.989 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:56.991 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:57.109 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-16T04:55:56.440801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":327495946,"num_objects":79964,"num_object_clones":0,"num_object_copies":159928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206670,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8828,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1865504,"kb_used_data":805596,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281250016,"statfs":{"total":289910292480,"available":288000016384,"internally_reserved":0,"allocated":824930304,"data_stored":824028734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1085262849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,5,5,3,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18968576,"num_objects":-4631,"num_object_clones":0,"num_object_copies":-9262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4631,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005333"},"pg_stats":[{"pgid":"2.f","version":"49'14855","reported_seq":14940,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.307187+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:54.307187+0000","last_peered":"2024-09-16T04:55:54.307187+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:54.307187+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20000768,"num_objects":4883,"num_object_clones":0,"num_object_copies":9766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14855,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1542}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15041","reported_seq":15162,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:53.842176+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:53.842176+0000","last_peered":"2024-09-16T04:55:53.842176+0000","last_clean":"2024-09-16T04:55:53.842176+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:53.842176+0000","last_undegraded":"2024-09-16T04:55:53.842176+0000","last_fullsized":"2024-09-16T04:55:53.842176+0000","mapping_epoch":35,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20287488,"num_objects":4953,"num_object_clones":0,"num_object_copies":9906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12010,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14905","reported_seq":15050,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.870004+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:53.870004+0000","last_peered":"2024-09-16T04:55:53.870004+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:53.870004+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283416,"num_objects":4953,"num_object_clones":0,"num_object_copies":9906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4953,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11644,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1634}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15210","reported_seq":15286,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:53.864844+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:53.864844+0000","last_peered":"2024-09-16T04:55:53.864844+0000","last_clean":"2024-09-16T04:55:53.864844+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:53.864844+0000","last_undegraded":"2024-09-16T04:55:53.864844+0000","last_fullsized":"2024-09-16T04:55:53.864844+0000","mapping_epoch":35,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21028864,"num_objects":5134,"num_object_clones":0,"num_object_copies":10268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15210,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15154","reported_seq":15378,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:54.298816+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:54.298816+0000","last_peered":"2024-09-16T04:55:54.298816+0000","last_clean":"2024-09-16T04:55:54.298816+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:54.298816+0000","last_undegraded":"2024-09-16T04:55:54.298816+0000","last_fullsized":"2024-09-16T04:55:54.298816+0000","mapping_epoch":34,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20439040,"num_objects":4990,"num_object_clones":0,"num_object_copies":9980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12186,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15288","reported_seq":15426,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.239499+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:54.239499+0000","last_peered":"2024-09-16T04:55:54.239499+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:54.239499+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20582400,"num_objects":5025,"num_object_clones":0,"num_object_copies":10050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12281,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14959","reported_seq":15113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.227979+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:54.227979+0000","last_peered":"2024-09-16T04:55:54.227979+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:54.227979+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11614,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15017","reported_seq":15238,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:54.271670+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:54.271670+0000","last_peered":"2024-09-16T04:55:54.271670+0000","last_clean":"2024-09-16T04:55:54.271670+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:54.271670+0000","last_undegraded":"2024-09-16T04:55:54.271670+0000","last_fullsized":"2024-09-16T04:55:54.271670+0000","mapping_epoch":34,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12087,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15065","reported_seq":15299,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.881320+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:53.881320+0000","last_peered":"2024-09-16T04:55:53.881320+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:53.881320+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20545536,"num_objects":5016,"num_object_clones":0,"num_object_copies":10032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11816,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15235","reported_seq":15371,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.238393+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:54.238393+0000","last_peered":"2024-09-16T04:55:54.238393+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:54.238393+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20410368,"num_objects":4983,"num_object_clones":0,"num_object_copies":9966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11890,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1518}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14990","reported_seq":19529,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.922522+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:53.922522+0000","last_peered":"2024-09-16T04:55:53.922522+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:53.922522+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20436178,"num_objects":4991,"num_object_clones":0,"num_object_copies":9982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14962,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2147,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3838}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15019","reported_seq":21881,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:53.890334+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:53.890334+0000","last_peered":"2024-09-16T04:55:53.890334+0000","last_clean":"2024-09-16T04:55:53.890334+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:53.890334+0000","last_undegraded":"2024-09-16T04:55:53.890334+0000","last_fullsized":"2024-09-16T04:55:53.890334+0000","mapping_epoch":46,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20312064,"num_objects":4959,"num_object_clones":0,"num_object_copies":9918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11742,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14984","reported_seq":15087,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.938248+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:53.938248+0000","last_peered":"2024-09-16T04:55:53.938248+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:53.938248+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20038249,"num_objects":4893,"num_object_clones":0,"num_object_copies":9786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14983,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1486}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15393","reported_seq":15501,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.771840+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:53.771840+0000","last_peered":"2024-09-16T04:55:53.771840+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:53.771840+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21051195,"num_objects":5142,"num_object_clones":0,"num_object_copies":10284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15390,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1628}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15092","reported_seq":21891,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:53.324167+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:53.324167+0000","last_peered":"2024-09-16T04:55:53.324167+0000","last_clean":"2024-09-16T04:55:53.324167+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:53.324167+0000","last_undegraded":"2024-09-16T04:55:53.324167+0000","last_fullsized":"2024-09-16T04:55:53.324167+0000","mapping_epoch":48,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20914176,"num_objects":5106,"num_object_clones":0,"num_object_copies":10212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11846,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15383","reported_seq":15529,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.289815+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:54.289815+0000","last_peered":"2024-09-16T04:55:54.289815+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:54.289815+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20615785,"num_objects":5034,"num_object_clones":0,"num_object_copies":10068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12152,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":327495927,"num_objects":79962,"num_object_clones":0,"num_object_copies":159924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206668,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8827,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":822464512,"data_stored":822390267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643472,"kb_used_data":282744,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93728368,"statfs":{"total":96636764160,"available":95977848832,"internally_reserved":0,"allocated":289529856,"data_stored":289246530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593804,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584524,"kb_used_data":241844,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93787316,"statfs":{"total":96636764160,"available":96038211584,"internally_reserved":0,"allocated":247648256,"data_stored":247309266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,1,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637508,"kb_used_data":281008,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93734332,"statfs":{"total":96636764160,"available":95983955968,"internally_reserved":0,"allocated":287752192,"data_stored":287472938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":287305728,"data_stored":287302249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":246218752,"data_stored":246155537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288940032,"data_stored":288932481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:57.111 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:57.890 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:57.891 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:57.989 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-16T04:55:56.440801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":327495946,"num_objects":79964,"num_object_clones":0,"num_object_copies":159928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206670,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8828,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1865504,"kb_used_data":805596,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281250016,"statfs":{"total":289910292480,"available":288000016384,"internally_reserved":0,"allocated":824930304,"data_stored":824028734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1085262849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,5,5,3,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18968576,"num_objects":-4631,"num_object_clones":0,"num_object_copies":-9262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4631,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005333"},"pg_stats":[{"pgid":"2.f","version":"49'14855","reported_seq":14940,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.307187+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:54.307187+0000","last_peered":"2024-09-16T04:55:54.307187+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:54.307187+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20000768,"num_objects":4883,"num_object_clones":0,"num_object_copies":9766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14855,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1542}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15041","reported_seq":15162,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:53.842176+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:53.842176+0000","last_peered":"2024-09-16T04:55:53.842176+0000","last_clean":"2024-09-16T04:55:53.842176+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:53.842176+0000","last_undegraded":"2024-09-16T04:55:53.842176+0000","last_fullsized":"2024-09-16T04:55:53.842176+0000","mapping_epoch":35,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20287488,"num_objects":4953,"num_object_clones":0,"num_object_copies":9906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12010,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14905","reported_seq":15050,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.870004+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:53.870004+0000","last_peered":"2024-09-16T04:55:53.870004+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:53.870004+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283416,"num_objects":4953,"num_object_clones":0,"num_object_copies":9906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4953,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11644,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1634}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15210","reported_seq":15286,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:53.864844+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:53.864844+0000","last_peered":"2024-09-16T04:55:53.864844+0000","last_clean":"2024-09-16T04:55:53.864844+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:53.864844+0000","last_undegraded":"2024-09-16T04:55:53.864844+0000","last_fullsized":"2024-09-16T04:55:53.864844+0000","mapping_epoch":35,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21028864,"num_objects":5134,"num_object_clones":0,"num_object_copies":10268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15210,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15154","reported_seq":15378,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:54.298816+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:54.298816+0000","last_peered":"2024-09-16T04:55:54.298816+0000","last_clean":"2024-09-16T04:55:54.298816+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:54.298816+0000","last_undegraded":"2024-09-16T04:55:54.298816+0000","last_fullsized":"2024-09-16T04:55:54.298816+0000","mapping_epoch":34,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20439040,"num_objects":4990,"num_object_clones":0,"num_object_copies":9980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12186,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15288","reported_seq":15426,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.239499+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:54.239499+0000","last_peered":"2024-09-16T04:55:54.239499+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:54.239499+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20582400,"num_objects":5025,"num_object_clones":0,"num_object_copies":10050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12281,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14959","reported_seq":15113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.227979+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:54.227979+0000","last_peered":"2024-09-16T04:55:54.227979+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:54.227979+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11614,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15017","reported_seq":15238,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:54.271670+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:54.271670+0000","last_peered":"2024-09-16T04:55:54.271670+0000","last_clean":"2024-09-16T04:55:54.271670+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:54.271670+0000","last_undegraded":"2024-09-16T04:55:54.271670+0000","last_fullsized":"2024-09-16T04:55:54.271670+0000","mapping_epoch":34,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12087,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15065","reported_seq":15299,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.881320+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:53.881320+0000","last_peered":"2024-09-16T04:55:53.881320+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:53.881320+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20545536,"num_objects":5016,"num_object_clones":0,"num_object_copies":10032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11816,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15235","reported_seq":15371,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.238393+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:54.238393+0000","last_peered":"2024-09-16T04:55:54.238393+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:54.238393+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20410368,"num_objects":4983,"num_object_clones":0,"num_object_copies":9966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11890,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1518}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14990","reported_seq":19529,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.922522+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:53.922522+0000","last_peered":"2024-09-16T04:55:53.922522+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:53.922522+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20436178,"num_objects":4991,"num_object_clones":0,"num_object_copies":9982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14962,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2147,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3838}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15019","reported_seq":21881,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:53.890334+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:53.890334+0000","last_peered":"2024-09-16T04:55:53.890334+0000","last_clean":"2024-09-16T04:55:53.890334+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:53.890334+0000","last_undegraded":"2024-09-16T04:55:53.890334+0000","last_fullsized":"2024-09-16T04:55:53.890334+0000","mapping_epoch":46,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20312064,"num_objects":4959,"num_object_clones":0,"num_object_copies":9918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11742,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14984","reported_seq":15087,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.938248+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:53.938248+0000","last_peered":"2024-09-16T04:55:53.938248+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:53.938248+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20038249,"num_objects":4893,"num_object_clones":0,"num_object_copies":9786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14983,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1486}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15393","reported_seq":15501,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.771840+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:53.771840+0000","last_peered":"2024-09-16T04:55:53.771840+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:53.771840+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21051195,"num_objects":5142,"num_object_clones":0,"num_object_copies":10284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15390,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1628}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15092","reported_seq":21891,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:53.324167+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:53.324167+0000","last_peered":"2024-09-16T04:55:53.324167+0000","last_clean":"2024-09-16T04:55:53.324167+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:53.324167+0000","last_undegraded":"2024-09-16T04:55:53.324167+0000","last_fullsized":"2024-09-16T04:55:53.324167+0000","mapping_epoch":48,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20914176,"num_objects":5106,"num_object_clones":0,"num_object_copies":10212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11846,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15383","reported_seq":15529,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.289815+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:54.289815+0000","last_peered":"2024-09-16T04:55:54.289815+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:54.289815+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20615785,"num_objects":5034,"num_object_clones":0,"num_object_copies":10068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12152,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":327495927,"num_objects":79962,"num_object_clones":0,"num_object_copies":159924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206668,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8827,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":822464512,"data_stored":822390267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643472,"kb_used_data":282744,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93728368,"statfs":{"total":96636764160,"available":95977848832,"internally_reserved":0,"allocated":289529856,"data_stored":289246530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593804,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584524,"kb_used_data":241844,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93787316,"statfs":{"total":96636764160,"available":96038211584,"internally_reserved":0,"allocated":247648256,"data_stored":247309266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,1,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637508,"kb_used_data":281008,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93734332,"statfs":{"total":96636764160,"available":95983955968,"internally_reserved":0,"allocated":287752192,"data_stored":287472938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":287305728,"data_stored":287302249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":246218752,"data_stored":246155537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288940032,"data_stored":288932481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:55:57.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:55:57.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'14855", 'reported_seq': 14940, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:54.307187+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:55:54.307187+0000', 'last_peered': '2024-09-16T04:55:54.307187+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:55:54.307187+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'4800", 'ondisk_log_start': "45'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20000768, 'num_objects': 4883, 'num_object_clones': 0, 'num_object_copies': 9766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4883, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14855, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1542}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:57.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:55:57.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'14905", 'reported_seq': 15050, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:53.870004+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:55:53.870004+0000', 'last_peered': '2024-09-16T04:55:53.870004+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:55:53.870004+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'4900", 'ondisk_log_start': "45'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20283416, 'num_objects': 4953, 'num_object_clones': 0, 'num_object_copies': 9906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4953, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11644, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1634}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:57.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:55:57.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'15288", 'reported_seq': 15426, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:54.239499+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:55:54.239499+0000', 'last_peered': '2024-09-16T04:55:54.239499+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:55:54.239499+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'5200", 'ondisk_log_start': "45'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20582400, 'num_objects': 5025, 'num_object_clones': 0, 'num_object_copies': 10050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5025, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12281, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1569}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:57.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:55:57.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'14959", 'reported_seq': 15113, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:54.227979+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:55:54.227979+0000', 'last_peered': '2024-09-16T04:55:54.227979+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:55:54.227979+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'4900", 'ondisk_log_start': "45'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20074496, 'num_objects': 4901, 'num_object_clones': 0, 'num_object_copies': 9802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4901, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11614, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1502}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:57.993 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:55:57.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'15065", 'reported_seq': 15299, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:53.881320+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:55:53.881320+0000', 'last_peered': '2024-09-16T04:55:53.881320+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:55:53.881320+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'5000", 'ondisk_log_start': "45'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20545536, 'num_objects': 5016, 'num_object_clones': 0, 'num_object_copies': 10032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11816, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1695}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:57.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:55:57.994 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'15235", 'reported_seq': 15371, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:54.238393+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:55:54.238393+0000', 'last_peered': '2024-09-16T04:55:54.238393+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:55:54.238393+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'5200", 'ondisk_log_start': "45'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20410368, 'num_objects': 4983, 'num_object_clones': 0, 'num_object_copies': 9966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4983, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11890, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1518}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:57.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:55:57.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'14990", 'reported_seq': 19529, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:53.922522+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:55:53.922522+0000', 'last_peered': '2024-09-16T04:55:53.922522+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:55:53.922522+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'4900", 'ondisk_log_start': "45'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20436178, 'num_objects': 4991, 'num_object_clones': 0, 'num_object_copies': 9982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1153, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4991, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14962, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2147, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3838}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:57.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:55:57.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'14984", 'reported_seq': 15087, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:53.938248+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:55:53.938248+0000', 'last_peered': '2024-09-16T04:55:53.938248+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:55:53.938248+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'4900", 'ondisk_log_start': "45'4900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20038249, 'num_objects': 4893, 'num_object_clones': 0, 'num_object_copies': 9786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14983, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:57.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:55:57.997 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'15393", 'reported_seq': 15501, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:53.771840+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:55:53.771840+0000', 'last_peered': '2024-09-16T04:55:53.771840+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:55:53.771840+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'5300", 'ondisk_log_start': "45'5300", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21051195, 'num_objects': 5142, 'num_object_clones': 0, 'num_object_copies': 10284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5142, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15390, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1628}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:55:57.997 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:55:57.997 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'15383", 'reported_seq': 15529, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:55:54.289815+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:55:54.289815+0000', 'last_peered': '2024-09-16T04:55:54.289815+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:55:54.289815+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'5300", 'ondisk_log_start': "45'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20615785, 'num_objects': 5034, 'num_object_clones': 0, 'num_object_copies': 10068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5034, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12152, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1510}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:55:57.997 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:55:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:58.998 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31915/159928 objects degraded (19.956%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":506,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":79964,"data_bytes":327495946,"bytes_used":1910276096,"bytes_avail":288000016384,"bytes_total":289910292480,"degraded_objects":31915,"degraded_total":159928,"degraded_ratio":0.19955855134810666,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":225,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":385},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:55:59.079 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:55:59.079 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:55:59.756 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:55:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:55:59.800 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-16T04:55:58.441244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":327495946,"num_objects":79964,"num_object_clones":0,"num_object_copies":159928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206670,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8828,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1865504,"kb_used_data":805596,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281250016,"statfs":{"total":289910292480,"available":288000016384,"internally_reserved":0,"allocated":824930304,"data_stored":824028734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1085262849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,5,5,3,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18968576,"num_objects":-4631,"num_object_clones":0,"num_object_copies":-9262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4631,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005286"},"pg_stats":[{"pgid":"2.f","version":"49'14855","reported_seq":14940,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.307187+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:54.307187+0000","last_peered":"2024-09-16T04:55:54.307187+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:54.307187+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'4800","ondisk_log_start":"45'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20000768,"num_objects":4883,"num_object_clones":0,"num_object_copies":9766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14855,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1542}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15041","reported_seq":15162,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:53.842176+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:53.842176+0000","last_peered":"2024-09-16T04:55:53.842176+0000","last_clean":"2024-09-16T04:55:53.842176+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:53.842176+0000","last_undegraded":"2024-09-16T04:55:53.842176+0000","last_fullsized":"2024-09-16T04:55:53.842176+0000","mapping_epoch":35,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20287488,"num_objects":4953,"num_object_clones":0,"num_object_copies":9906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12010,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'14905","reported_seq":15050,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.870004+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:53.870004+0000","last_peered":"2024-09-16T04:55:53.870004+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:53.870004+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20283416,"num_objects":4953,"num_object_clones":0,"num_object_copies":9906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4953,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11644,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1634}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15210","reported_seq":15286,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:53.864844+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:53.864844+0000","last_peered":"2024-09-16T04:55:53.864844+0000","last_clean":"2024-09-16T04:55:53.864844+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:53.864844+0000","last_undegraded":"2024-09-16T04:55:53.864844+0000","last_fullsized":"2024-09-16T04:55:53.864844+0000","mapping_epoch":35,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21028864,"num_objects":5134,"num_object_clones":0,"num_object_copies":10268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15210,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15154","reported_seq":15378,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:54.298816+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:54.298816+0000","last_peered":"2024-09-16T04:55:54.298816+0000","last_clean":"2024-09-16T04:55:54.298816+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:54.298816+0000","last_undegraded":"2024-09-16T04:55:54.298816+0000","last_fullsized":"2024-09-16T04:55:54.298816+0000","mapping_epoch":34,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20439040,"num_objects":4990,"num_object_clones":0,"num_object_copies":9980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12186,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15288","reported_seq":15426,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.239499+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:54.239499+0000","last_peered":"2024-09-16T04:55:54.239499+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:54.239499+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20582400,"num_objects":5025,"num_object_clones":0,"num_object_copies":10050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12281,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'14959","reported_seq":15113,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.227979+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:54.227979+0000","last_peered":"2024-09-16T04:55:54.227979+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:54.227979+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11614,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15017","reported_seq":15238,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:54.271670+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:54.271670+0000","last_peered":"2024-09-16T04:55:54.271670+0000","last_clean":"2024-09-16T04:55:54.271670+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:54.271670+0000","last_undegraded":"2024-09-16T04:55:54.271670+0000","last_fullsized":"2024-09-16T04:55:54.271670+0000","mapping_epoch":34,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12087,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15065","reported_seq":15299,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.881320+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:53.881320+0000","last_peered":"2024-09-16T04:55:53.881320+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:53.881320+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20545536,"num_objects":5016,"num_object_clones":0,"num_object_copies":10032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11816,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15235","reported_seq":15371,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.238393+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:54.238393+0000","last_peered":"2024-09-16T04:55:54.238393+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:54.238393+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20410368,"num_objects":4983,"num_object_clones":0,"num_object_copies":9966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11890,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1518}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'14990","reported_seq":19529,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.922522+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:53.922522+0000","last_peered":"2024-09-16T04:55:53.922522+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:53.922522+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20436178,"num_objects":4991,"num_object_clones":0,"num_object_copies":9982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14962,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2147,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3838}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15019","reported_seq":21881,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:53.890334+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:53.890334+0000","last_peered":"2024-09-16T04:55:53.890334+0000","last_clean":"2024-09-16T04:55:53.890334+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:53.890334+0000","last_undegraded":"2024-09-16T04:55:53.890334+0000","last_fullsized":"2024-09-16T04:55:53.890334+0000","mapping_epoch":46,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20312064,"num_objects":4959,"num_object_clones":0,"num_object_copies":9918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11742,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'14984","reported_seq":15087,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.938248+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:53.938248+0000","last_peered":"2024-09-16T04:55:53.938248+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:53.938248+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'4900","ondisk_log_start":"45'4900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20038249,"num_objects":4893,"num_object_clones":0,"num_object_copies":9786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14983,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1486}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15393","reported_seq":15501,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:53.771840+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:53.771840+0000","last_peered":"2024-09-16T04:55:53.771840+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:53.771840+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21051195,"num_objects":5142,"num_object_clones":0,"num_object_copies":10284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15390,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1628}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15092","reported_seq":21891,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:53.324167+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:53.324167+0000","last_peered":"2024-09-16T04:55:53.324167+0000","last_clean":"2024-09-16T04:55:53.324167+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:53.324167+0000","last_undegraded":"2024-09-16T04:55:53.324167+0000","last_fullsized":"2024-09-16T04:55:53.324167+0000","mapping_epoch":48,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20914176,"num_objects":5106,"num_object_clones":0,"num_object_copies":10212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11846,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567763+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567763+0000","last_peered":"2024-09-16T04:54:05.567763+0000","last_clean":"2024-09-16T04:54:05.567763+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567763+0000","last_undegraded":"2024-09-16T04:54:05.567763+0000","last_fullsized":"2024-09-16T04:54:05.567763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15383","reported_seq":15529,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:54.289815+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:54.289815+0000","last_peered":"2024-09-16T04:55:54.289815+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:54.289815+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20615785,"num_objects":5034,"num_object_clones":0,"num_object_copies":10068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12152,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":327495927,"num_objects":79962,"num_object_clones":0,"num_object_copies":159924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206668,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8827,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":822464512,"data_stored":822390267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643472,"kb_used_data":282744,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93728368,"statfs":{"total":96636764160,"available":95977848832,"internally_reserved":0,"allocated":289529856,"data_stored":289246530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593804,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584524,"kb_used_data":241844,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93787316,"statfs":{"total":96636764160,"available":96038211584,"internally_reserved":0,"allocated":247648256,"data_stored":247309266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,1,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637508,"kb_used_data":281008,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93734332,"statfs":{"total":96636764160,"available":95983955968,"internally_reserved":0,"allocated":287752192,"data_stored":287472938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":287305728,"data_stored":287302249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":246218752,"data_stored":246155537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288940032,"data_stored":288932481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:02.803 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:03.663 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:03.765 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-16T04:56:02.444607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317911306,"num_objects":77624,"num_object_clones":0,"num_object_copies":155248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77624,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209010,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8954,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1847948,"kb_used_data":788056,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281267572,"statfs":{"total":289910292480,"available":288017993728,"internally_reserved":0,"allocated":806969344,"data_stored":806067774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1085262849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,2,8,12,17,2,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19066880,"num_objects":-4655,"num_object_clones":0,"num_object_copies":-9310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-237,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007455"},"pg_stats":[{"pgid":"2.f","version":"49'15010","reported_seq":15095,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:59.336600+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:55:59.336600+0000","last_peered":"2024-09-16T04:55:59.336600+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:55:59.336600+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19365888,"num_objects":4728,"num_object_clones":0,"num_object_copies":9456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15010,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1387}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15199","reported_seq":15321,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:58.920618+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:55:58.920618+0000","last_peered":"2024-09-16T04:55:58.920618+0000","last_clean":"2024-09-16T04:55:58.920618+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:55:58.920618+0000","last_undegraded":"2024-09-16T04:55:58.920618+0000","last_fullsized":"2024-09-16T04:55:58.920618+0000","mapping_epoch":35,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19640320,"num_objects":4795,"num_object_clones":0,"num_object_copies":9590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12168,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15052","reported_seq":15197,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:58.763211+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:55:58.763211+0000","last_peered":"2024-09-16T04:55:58.763211+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:55:58.763211+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19681304,"num_objects":4806,"num_object_clones":0,"num_object_copies":9612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4806,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11791,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1487}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15367","reported_seq":15443,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:58.964840+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:55:58.964840+0000","last_peered":"2024-09-16T04:55:58.964840+0000","last_clean":"2024-09-16T04:55:58.964840+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:55:58.964840+0000","last_undegraded":"2024-09-16T04:55:58.964840+0000","last_fullsized":"2024-09-16T04:55:58.964840+0000","mapping_epoch":35,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20385792,"num_objects":4977,"num_object_clones":0,"num_object_copies":9954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15367,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15284","reported_seq":15508,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:59.330380+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:55:59.330380+0000","last_peered":"2024-09-16T04:55:59.330380+0000","last_clean":"2024-09-16T04:55:59.330380+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:55:59.330380+0000","last_undegraded":"2024-09-16T04:55:59.330380+0000","last_fullsized":"2024-09-16T04:55:59.330380+0000","mapping_epoch":34,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19906560,"num_objects":4860,"num_object_clones":0,"num_object_copies":9720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12316,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15440","reported_seq":15575,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:59.258348+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:55:59.258348+0000","last_peered":"2024-09-16T04:55:59.258348+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:55:59.258348+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19959808,"num_objects":4873,"num_object_clones":0,"num_object_copies":9746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12433,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15088","reported_seq":15242,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:59.309962+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:55:59.309962+0000","last_peered":"2024-09-16T04:55:59.309962+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:55:59.309962+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5000","ondisk_log_start":"45'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19546112,"num_objects":4772,"num_object_clones":0,"num_object_copies":9544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11743,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1373}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15152","reported_seq":15374,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:59.273218+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:55:59.273218+0000","last_peered":"2024-09-16T04:55:59.273218+0000","last_clean":"2024-09-16T04:55:59.273218+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:55:59.273218+0000","last_undegraded":"2024-09-16T04:55:59.273218+0000","last_fullsized":"2024-09-16T04:55:59.273218+0000","mapping_epoch":34,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19922944,"num_objects":4864,"num_object_clones":0,"num_object_copies":9728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12222,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15204","reported_seq":15437,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:58.920671+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:55:58.920671+0000","last_peered":"2024-09-16T04:55:58.920671+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:55:58.920671+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19976192,"num_objects":4877,"num_object_clones":0,"num_object_copies":9754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11955,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1556}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15376","reported_seq":15510,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:59.259864+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:55:59.259864+0000","last_peered":"2024-09-16T04:55:59.259864+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:55:59.259864+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19832832,"num_objects":4842,"num_object_clones":0,"num_object_copies":9684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12031,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1377}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15123","reported_seq":19914,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:58.948796+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:55:58.948796+0000","last_peered":"2024-09-16T04:55:58.948796+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:55:58.948796+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19891410,"num_objects":4858,"num_object_clones":0,"num_object_copies":9716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15095,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2273,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3831}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15166","reported_seq":22028,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:58.850800+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:55:58.850800+0000","last_peered":"2024-09-16T04:55:58.850800+0000","last_clean":"2024-09-16T04:55:58.850800+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:55:58.850800+0000","last_undegraded":"2024-09-16T04:55:58.850800+0000","last_fullsized":"2024-09-16T04:55:58.850800+0000","mapping_epoch":46,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19709952,"num_objects":4812,"num_object_clones":0,"num_object_copies":9624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11889,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15128","reported_seq":15231,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:58.966436+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:55:58.966436+0000","last_peered":"2024-09-16T04:55:58.966436+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:55:58.966436+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19448425,"num_objects":4749,"num_object_clones":0,"num_object_copies":9498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15127,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1342}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15538","reported_seq":15646,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:58.937287+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:55:58.937287+0000","last_peered":"2024-09-16T04:55:58.937287+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:55:58.937287+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20457275,"num_objects":4997,"num_object_clones":0,"num_object_copies":9994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15535,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1483}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15256","reported_seq":22054,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:55:58.423767+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:55:58.423767+0000","last_peered":"2024-09-16T04:55:58.423767+0000","last_clean":"2024-09-16T04:55:58.423767+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:55:58.423767+0000","last_undegraded":"2024-09-16T04:55:58.423767+0000","last_fullsized":"2024-09-16T04:55:58.423767+0000","mapping_epoch":48,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20242432,"num_objects":4942,"num_object_clones":0,"num_object_copies":9884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12010,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15547","reported_seq":15693,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:55:59.196309+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:55:59.196309+0000","last_peered":"2024-09-16T04:55:59.196309+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:55:59.196309+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19944041,"num_objects":4870,"num_object_clones":0,"num_object_copies":9740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12316,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1346}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":317911287,"num_objects":77622,"num_object_clones":0,"num_object_copies":155244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31789,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209008,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8953,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":803381248,"data_stored":803307003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160830,"ondisk_log_size":160830,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":641900,"kb_used_data":277020,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93729940,"statfs":{"total":96636764160,"available":95979458560,"internally_reserved":0,"allocated":283668480,"data_stored":283385154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,5,7,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593805,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569248,"kb_used_data":234896,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93802592,"statfs":{"total":96636764160,"available":96053854208,"internally_reserved":0,"allocated":240533504,"data_stored":240194514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,2,2,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636800,"kb_used_data":276140,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93735040,"statfs":{"total":96636764160,"available":95984680960,"internally_reserved":0,"allocated":282767360,"data_stored":282488106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,1,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":281309184,"data_stored":281305705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":239001600,"data_stored":238938385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":283070464,"data_stored":283062913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:03.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:04.630 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:04.631 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:04.705 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-16T04:56:04.445051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":308269322,"num_objects":75270,"num_object_clones":0,"num_object_copies":150540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":211364,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9071,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160887,"ondisk_log_size":160887,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1824964,"kb_used_data":769236,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281290556,"statfs":{"total":289910292480,"available":288041529344,"internally_reserved":0,"allocated":787697664,"data_stored":786796094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1081003009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,4,3,12,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28708864,"num_objects":-7009,"num_object_clones":0,"num_object_copies":-14018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-354,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7009,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005220"},"pg_stats":[{"pgid":"2.f","version":"49'15156","reported_seq":15241,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:04.322671+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:04.322671+0000","last_peered":"2024-09-16T04:56:04.322671+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:04.322671+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18767872,"num_objects":4582,"num_object_clones":0,"num_object_copies":9164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15156,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1241}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15359","reported_seq":15481,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:03.979272+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:03.979272+0000","last_peered":"2024-09-16T04:56:03.979272+0000","last_clean":"2024-09-16T04:56:03.979272+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:03.979272+0000","last_undegraded":"2024-09-16T04:56:03.979272+0000","last_fullsized":"2024-09-16T04:56:03.979272+0000","mapping_epoch":35,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18984960,"num_objects":4635,"num_object_clones":0,"num_object_copies":9270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12328,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15194","reported_seq":15339,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:03.960939+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:03.960939+0000","last_peered":"2024-09-16T04:56:03.960939+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:03.960939+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19099672,"num_objects":4664,"num_object_clones":0,"num_object_copies":9328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11933,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1345}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15513","reported_seq":15589,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:04.008617+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:04.008617+0000","last_peered":"2024-09-16T04:56:04.008617+0000","last_clean":"2024-09-16T04:56:04.008617+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:04.008617+0000","last_undegraded":"2024-09-16T04:56:04.008617+0000","last_fullsized":"2024-09-16T04:56:04.008617+0000","mapping_epoch":35,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19787776,"num_objects":4831,"num_object_clones":0,"num_object_copies":9662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15513,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15428","reported_seq":15652,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:04.328297+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:04.328297+0000","last_peered":"2024-09-16T04:56:04.328297+0000","last_clean":"2024-09-16T04:56:04.328297+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:04.328297+0000","last_undegraded":"2024-09-16T04:56:04.328297+0000","last_fullsized":"2024-09-16T04:56:04.328297+0000","mapping_epoch":34,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19316736,"num_objects":4716,"num_object_clones":0,"num_object_copies":9432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12460,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15587","reported_seq":15722,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:04.333846+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:04.333846+0000","last_peered":"2024-09-16T04:56:04.333846+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:04.333846+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19357696,"num_objects":4726,"num_object_clones":0,"num_object_copies":9452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12580,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1270}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15225","reported_seq":15379,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:04.223482+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:04.223482+0000","last_peered":"2024-09-16T04:56:04.223482+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:04.223482+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18984960,"num_objects":4635,"num_object_clones":0,"num_object_copies":9270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11880,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15295","reported_seq":15517,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:04.311867+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:04.311867+0000","last_peered":"2024-09-16T04:56:04.311867+0000","last_clean":"2024-09-16T04:56:04.311867+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:04.311867+0000","last_undegraded":"2024-09-16T04:56:04.311867+0000","last_fullsized":"2024-09-16T04:56:04.311867+0000","mapping_epoch":34,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19337216,"num_objects":4721,"num_object_clones":0,"num_object_copies":9442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12365,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15346","reported_seq":15580,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:03.960379+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:03.960379+0000","last_peered":"2024-09-16T04:56:03.960379+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:03.960379+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19394560,"num_objects":4735,"num_object_clones":0,"num_object_copies":9470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12097,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1414}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15530","reported_seq":15666,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:04.273986+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:04.273986+0000","last_peered":"2024-09-16T04:56:04.273986+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:04.273986+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19202048,"num_objects":4688,"num_object_clones":0,"num_object_copies":9376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12185,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1223}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15277","reported_seq":20301,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:03.988216+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:03.988216+0000","last_peered":"2024-09-16T04:56:03.988216+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:03.988216+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19260626,"num_objects":4704,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15249,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2390,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3794}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15325","reported_seq":22187,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:03.923223+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:03.923223+0000","last_peered":"2024-09-16T04:56:03.923223+0000","last_clean":"2024-09-16T04:56:03.923223+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:03.923223+0000","last_undegraded":"2024-09-16T04:56:03.923223+0000","last_fullsized":"2024-09-16T04:56:03.923223+0000","mapping_epoch":46,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19058688,"num_objects":4653,"num_object_clones":0,"num_object_copies":9306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12048,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15270","reported_seq":15373,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:03.915901+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:03.915901+0000","last_peered":"2024-09-16T04:56:03.915901+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:03.915901+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18866793,"num_objects":4607,"num_object_clones":0,"num_object_copies":9214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15269,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1200}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15682","reported_seq":15789,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:03.996980+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:03.996980+0000","last_peered":"2024-09-16T04:56:03.996980+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:03.996980+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19867451,"num_objects":4853,"num_object_clones":0,"num_object_copies":9706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15679,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1339}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15405","reported_seq":22204,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:03.465342+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:03.465342+0000","last_peered":"2024-09-16T04:56:03.465342+0000","last_clean":"2024-09-16T04:56:03.465342+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:03.465342+0000","last_undegraded":"2024-09-16T04:56:03.465342+0000","last_fullsized":"2024-09-16T04:56:03.465342+0000","mapping_epoch":48,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19632128,"num_objects":4793,"num_object_clones":0,"num_object_copies":9586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12159,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15692","reported_seq":15838,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:04.242468+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:04.242468+0000","last_peered":"2024-09-16T04:56:04.242468+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:04.242468+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19350121,"num_objects":4725,"num_object_clones":0,"num_object_copies":9450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12461,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":308269303,"num_objects":75268,"num_object_clones":0,"num_object_copies":150536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75268,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":211362,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9070,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":784019456,"data_stored":783945211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160884,"ondisk_log_size":160884,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623600,"kb_used_data":271200,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93748240,"statfs":{"total":96636764160,"available":95998197760,"internally_reserved":0,"allocated":277708800,"data_stored":277425474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593806,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566320,"kb_used_data":227808,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93805520,"statfs":{"total":96636764160,"available":96056852480,"internally_reserved":0,"allocated":233275392,"data_stored":232936402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,7,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635044,"kb_used_data":270228,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93736796,"statfs":{"total":96636764160,"available":95986479104,"internally_reserved":0,"allocated":276713472,"data_stored":276434218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":275247104,"data_stored":275243625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231727104,"data_stored":231663889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":277045248,"data_stored":277037697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:04.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:56:04.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'15156", 'reported_seq': 15241, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:04.322671+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:56:04.322671+0000', 'last_peered': '2024-09-16T04:56:04.322671+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:56:04.322671+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'5100", 'ondisk_log_start': "45'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18767872, 'num_objects': 4582, 'num_object_clones': 0, 'num_object_copies': 9164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4582, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15156, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1241}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:04.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:56:04.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'15194", 'reported_seq': 15339, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:03.960939+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:56:03.960939+0000', 'last_peered': '2024-09-16T04:56:03.960939+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:56:03.960939+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'5100", 'ondisk_log_start': "45'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19099672, 'num_objects': 4664, 'num_object_clones': 0, 'num_object_copies': 9328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4664, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11933, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1345}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:04.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:56:04.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'15587", 'reported_seq': 15722, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:04.333846+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:56:04.333846+0000', 'last_peered': '2024-09-16T04:56:04.333846+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:56:04.333846+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'5500", 'ondisk_log_start': "45'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19357696, 'num_objects': 4726, 'num_object_clones': 0, 'num_object_copies': 9452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12580, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1270}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:04.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:56:04.707 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'15225", 'reported_seq': 15379, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:04.223482+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:56:04.223482+0000', 'last_peered': '2024-09-16T04:56:04.223482+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:56:04.223482+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'5200", 'ondisk_log_start': "45'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18984960, 'num_objects': 4635, 'num_object_clones': 0, 'num_object_copies': 9270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4635, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11880, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1236}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:04.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:56:04.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'15346", 'reported_seq': 15580, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:03.960379+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:56:03.960379+0000', 'last_peered': '2024-09-16T04:56:03.960379+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:56:03.960379+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'5300", 'ondisk_log_start': "45'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19394560, 'num_objects': 4735, 'num_object_clones': 0, 'num_object_copies': 9470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4735, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12097, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1414}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:04.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:56:04.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'15530", 'reported_seq': 15666, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:04.273986+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:56:04.273986+0000', 'last_peered': '2024-09-16T04:56:04.273986+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:56:04.273986+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'5500", 'ondisk_log_start': "45'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19202048, 'num_objects': 4688, 'num_object_clones': 0, 'num_object_copies': 9376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4688, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12185, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1223}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:04.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:56:04.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'15277", 'reported_seq': 20301, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:03.988216+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:56:03.988216+0000', 'last_peered': '2024-09-16T04:56:03.988216+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:56:03.988216+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'5200", 'ondisk_log_start': "45'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19260626, 'num_objects': 4704, 'num_object_clones': 0, 'num_object_copies': 9408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 910, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4704, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15249, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2390, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3794}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:04.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:56:04.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'15270", 'reported_seq': 15373, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:03.915901+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:56:03.915901+0000', 'last_peered': '2024-09-16T04:56:03.915901+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:56:03.915901+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'5200", 'ondisk_log_start': "45'5200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18866793, 'num_objects': 4607, 'num_object_clones': 0, 'num_object_copies': 9214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4607, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15269, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1200}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:04.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:56:04.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'15682", 'reported_seq': 15789, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:03.996980+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:56:03.996980+0000', 'last_peered': '2024-09-16T04:56:03.996980+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:56:03.996980+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'5600", 'ondisk_log_start': "45'5600", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19867451, 'num_objects': 4853, 'num_object_clones': 0, 'num_object_copies': 9706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15679, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1339}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:04.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:56:04.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'15692", 'reported_seq': 15838, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:04.242468+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:56:04.242468+0000', 'last_peered': '2024-09-16T04:56:04.242468+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:56:04.242468+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'5600", 'ondisk_log_start': "45'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19350121, 'num_objects': 4725, 'num_object_clones': 0, 'num_object_copies': 9450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4725, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12461, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1201}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:04.709 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:56:05.779 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:05.779 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31672/150540 objects degraded (21.039%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":513,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":75270,"data_bytes":308269322,"bytes_used":1868763136,"bytes_avail":288041529344,"bytes_total":289910292480,"degraded_objects":31672,"degraded_total":150540,"degraded_ratio":0.21038926531154512,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":354,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":583},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:56:05.884 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:56:05.884 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:06.546 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:06.603 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-16T04:56:06.445526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":308269322,"num_objects":75270,"num_object_clones":0,"num_object_copies":150540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":211364,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9071,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160887,"ondisk_log_size":160887,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1824964,"kb_used_data":769236,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281290556,"statfs":{"total":289910292480,"available":288041529344,"internally_reserved":0,"allocated":787697664,"data_stored":786796094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1081003009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,4,3,12,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19226624,"num_objects":-4694,"num_object_clones":0,"num_object_copies":-9388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4694,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005233"},"pg_stats":[{"pgid":"2.f","version":"49'15156","reported_seq":15241,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:04.322671+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:04.322671+0000","last_peered":"2024-09-16T04:56:04.322671+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:04.322671+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18767872,"num_objects":4582,"num_object_clones":0,"num_object_copies":9164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15156,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1241}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15359","reported_seq":15481,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:03.979272+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:03.979272+0000","last_peered":"2024-09-16T04:56:03.979272+0000","last_clean":"2024-09-16T04:56:03.979272+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:03.979272+0000","last_undegraded":"2024-09-16T04:56:03.979272+0000","last_fullsized":"2024-09-16T04:56:03.979272+0000","mapping_epoch":35,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18984960,"num_objects":4635,"num_object_clones":0,"num_object_copies":9270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12328,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15194","reported_seq":15339,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:03.960939+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:03.960939+0000","last_peered":"2024-09-16T04:56:03.960939+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:03.960939+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5100","ondisk_log_start":"45'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19099672,"num_objects":4664,"num_object_clones":0,"num_object_copies":9328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11933,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1345}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15513","reported_seq":15589,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:04.008617+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:04.008617+0000","last_peered":"2024-09-16T04:56:04.008617+0000","last_clean":"2024-09-16T04:56:04.008617+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:04.008617+0000","last_undegraded":"2024-09-16T04:56:04.008617+0000","last_fullsized":"2024-09-16T04:56:04.008617+0000","mapping_epoch":35,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19787776,"num_objects":4831,"num_object_clones":0,"num_object_copies":9662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15513,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15428","reported_seq":15652,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:04.328297+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:04.328297+0000","last_peered":"2024-09-16T04:56:04.328297+0000","last_clean":"2024-09-16T04:56:04.328297+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:04.328297+0000","last_undegraded":"2024-09-16T04:56:04.328297+0000","last_fullsized":"2024-09-16T04:56:04.328297+0000","mapping_epoch":34,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19316736,"num_objects":4716,"num_object_clones":0,"num_object_copies":9432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12460,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15587","reported_seq":15722,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:04.333846+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:04.333846+0000","last_peered":"2024-09-16T04:56:04.333846+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:04.333846+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19357696,"num_objects":4726,"num_object_clones":0,"num_object_copies":9452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12580,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1270}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15225","reported_seq":15379,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:04.223482+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:04.223482+0000","last_peered":"2024-09-16T04:56:04.223482+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:04.223482+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18984960,"num_objects":4635,"num_object_clones":0,"num_object_copies":9270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11880,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15295","reported_seq":15517,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:04.311867+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:04.311867+0000","last_peered":"2024-09-16T04:56:04.311867+0000","last_clean":"2024-09-16T04:56:04.311867+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:04.311867+0000","last_undegraded":"2024-09-16T04:56:04.311867+0000","last_fullsized":"2024-09-16T04:56:04.311867+0000","mapping_epoch":34,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19337216,"num_objects":4721,"num_object_clones":0,"num_object_copies":9442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12365,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15346","reported_seq":15580,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:03.960379+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:03.960379+0000","last_peered":"2024-09-16T04:56:03.960379+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:03.960379+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19394560,"num_objects":4735,"num_object_clones":0,"num_object_copies":9470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12097,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1414}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15530","reported_seq":15666,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:04.273986+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:04.273986+0000","last_peered":"2024-09-16T04:56:04.273986+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:04.273986+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19202048,"num_objects":4688,"num_object_clones":0,"num_object_copies":9376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12185,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1223}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15277","reported_seq":20301,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:03.988216+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:03.988216+0000","last_peered":"2024-09-16T04:56:03.988216+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:03.988216+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19260626,"num_objects":4704,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15249,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2390,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3794}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15325","reported_seq":22187,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:03.923223+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:03.923223+0000","last_peered":"2024-09-16T04:56:03.923223+0000","last_clean":"2024-09-16T04:56:03.923223+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:03.923223+0000","last_undegraded":"2024-09-16T04:56:03.923223+0000","last_fullsized":"2024-09-16T04:56:03.923223+0000","mapping_epoch":46,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19058688,"num_objects":4653,"num_object_clones":0,"num_object_copies":9306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12048,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15270","reported_seq":15373,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:03.915901+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:03.915901+0000","last_peered":"2024-09-16T04:56:03.915901+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:03.915901+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18866793,"num_objects":4607,"num_object_clones":0,"num_object_copies":9214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15269,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1200}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15682","reported_seq":15789,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:03.996980+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:03.996980+0000","last_peered":"2024-09-16T04:56:03.996980+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:03.996980+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19867451,"num_objects":4853,"num_object_clones":0,"num_object_copies":9706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15679,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1339}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15405","reported_seq":22204,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:03.465342+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:03.465342+0000","last_peered":"2024-09-16T04:56:03.465342+0000","last_clean":"2024-09-16T04:56:03.465342+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:03.465342+0000","last_undegraded":"2024-09-16T04:56:03.465342+0000","last_fullsized":"2024-09-16T04:56:03.465342+0000","mapping_epoch":48,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19632128,"num_objects":4793,"num_object_clones":0,"num_object_copies":9586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12159,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567640+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567640+0000","last_peered":"2024-09-16T04:54:05.567640+0000","last_clean":"2024-09-16T04:54:05.567640+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567640+0000","last_undegraded":"2024-09-16T04:54:05.567640+0000","last_fullsized":"2024-09-16T04:54:05.567640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15692","reported_seq":15838,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:04.242468+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:04.242468+0000","last_peered":"2024-09-16T04:56:04.242468+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:04.242468+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19350121,"num_objects":4725,"num_object_clones":0,"num_object_copies":9450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12461,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":308269303,"num_objects":75268,"num_object_clones":0,"num_object_copies":150536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75268,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":211362,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9070,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":784019456,"data_stored":783945211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160884,"ondisk_log_size":160884,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623600,"kb_used_data":271200,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93748240,"statfs":{"total":96636764160,"available":95998197760,"internally_reserved":0,"allocated":277708800,"data_stored":277425474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593806,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566320,"kb_used_data":227808,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93805520,"statfs":{"total":96636764160,"available":96056852480,"internally_reserved":0,"allocated":233275392,"data_stored":232936402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,7,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635044,"kb_used_data":270228,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93736796,"statfs":{"total":96636764160,"available":95986479104,"internally_reserved":0,"allocated":276713472,"data_stored":276434218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":275247104,"data_stored":275243625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231727104,"data_stored":231663889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":277045248,"data_stored":277037697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:09.606 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:10.528 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:10.578 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-16T04:56:10.447965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":298606858,"num_objects":72911,"num_object_clones":0,"num_object_copies":145822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31547,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72911,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213723,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9196,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1845768,"kb_used_data":749768,"kb_used_omap":12,"kb_used_meta":1095923,"kb_avail":281269752,"statfs":{"total":289910292480,"available":288020226048,"internally_reserved":0,"allocated":767762432,"data_stored":766860862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1122225153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,2,4,10,11,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28889088,"num_objects":-7053,"num_object_clones":0,"num_object_copies":-14106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7053,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006720"},"pg_stats":[{"pgid":"2.f","version":"49'15295","reported_seq":15380,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.400626+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:09.400626+0000","last_peered":"2024-09-16T04:56:09.400626+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:09.400626+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18198528,"num_objects":4443,"num_object_clones":0,"num_object_copies":8886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15295,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1102}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15532","reported_seq":15654,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.031770+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:09.031770+0000","last_peered":"2024-09-16T04:56:09.031770+0000","last_clean":"2024-09-16T04:56:09.031770+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:09.031770+0000","last_undegraded":"2024-09-16T04:56:09.031770+0000","last_fullsized":"2024-09-16T04:56:09.031770+0000","mapping_epoch":35,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18276352,"num_objects":4462,"num_object_clones":0,"num_object_copies":8924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12501,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15334","reported_seq":15479,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:08.997430+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:08.997430+0000","last_peered":"2024-09-16T04:56:08.997430+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:08.997430+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18526232,"num_objects":4524,"num_object_clones":0,"num_object_copies":9048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12073,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15651","reported_seq":15727,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.030547+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:09.030547+0000","last_peered":"2024-09-16T04:56:09.030547+0000","last_clean":"2024-09-16T04:56:09.030547+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:09.030547+0000","last_undegraded":"2024-09-16T04:56:09.030547+0000","last_fullsized":"2024-09-16T04:56:09.030547+0000","mapping_epoch":35,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19222528,"num_objects":4693,"num_object_clones":0,"num_object_copies":9386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15651,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15572","reported_seq":15796,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.410015+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:09.410015+0000","last_peered":"2024-09-16T04:56:09.410015+0000","last_clean":"2024-09-16T04:56:09.410015+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:09.410015+0000","last_undegraded":"2024-09-16T04:56:09.410015+0000","last_fullsized":"2024-09-16T04:56:09.410015+0000","mapping_epoch":34,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18726912,"num_objects":4572,"num_object_clones":0,"num_object_copies":9144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12604,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15743","reported_seq":15882,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.380751+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:09.380751+0000","last_peered":"2024-09-16T04:56:09.380751+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:09.380751+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18718720,"num_objects":4570,"num_object_clones":0,"num_object_copies":9140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12736,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15374","reported_seq":15528,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.336089+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:09.336089+0000","last_peered":"2024-09-16T04:56:09.336089+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:09.336089+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18374656,"num_objects":4486,"num_object_clones":0,"num_object_copies":8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12029,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15435","reported_seq":15657,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.366746+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:09.366746+0000","last_peered":"2024-09-16T04:56:09.366746+0000","last_clean":"2024-09-16T04:56:09.366746+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:09.366746+0000","last_undegraded":"2024-09-16T04:56:09.366746+0000","last_fullsized":"2024-09-16T04:56:09.366746+0000","mapping_epoch":34,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18763776,"num_objects":4581,"num_object_clones":0,"num_object_copies":9162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12505,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15485","reported_seq":15719,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:08.952601+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:08.952601+0000","last_peered":"2024-09-16T04:56:08.952601+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:08.952601+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18825216,"num_objects":4596,"num_object_clones":0,"num_object_copies":9192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12236,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1275}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15701","reported_seq":15837,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.364389+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:09.364389+0000","last_peered":"2024-09-16T04:56:09.364389+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:09.364389+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18501632,"num_objects":4517,"num_object_clones":0,"num_object_copies":9034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12356,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1052}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15414","reported_seq":20688,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.016959+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:09.016959+0000","last_peered":"2024-09-16T04:56:09.016959+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:09.016959+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18699474,"num_objects":4567,"num_object_clones":0,"num_object_copies":9134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15386,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2515,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3782}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15481","reported_seq":22343,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.018885+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:09.018885+0000","last_peered":"2024-09-16T04:56:09.018885+0000","last_clean":"2024-09-16T04:56:09.018885+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:09.018885+0000","last_undegraded":"2024-09-16T04:56:09.018885+0000","last_fullsized":"2024-09-16T04:56:09.018885+0000","mapping_epoch":46,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18419712,"num_objects":4497,"num_object_clones":0,"num_object_copies":8994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12204,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15413","reported_seq":15515,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.017835+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:09.017835+0000","last_peered":"2024-09-16T04:56:09.017835+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:09.017835+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18281065,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15412,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1057}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15825","reported_seq":15934,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.002807+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:09.002807+0000","last_peered":"2024-09-16T04:56:09.002807+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:09.002807+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19281723,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15822,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1196}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15546","reported_seq":22345,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:08.542608+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:08.542608+0000","last_peered":"2024-09-16T04:56:08.542608+0000","last_clean":"2024-09-16T04:56:08.542608+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:08.542608+0000","last_undegraded":"2024-09-16T04:56:08.542608+0000","last_fullsized":"2024-09-16T04:56:08.542608+0000","mapping_epoch":48,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19054592,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12300,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15842","reported_seq":15988,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.389022+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:09.389022+0000","last_peered":"2024-09-16T04:56:09.389022+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:09.389022+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18735721,"num_objects":4575,"num_object_clones":0,"num_object_copies":9150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12611,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":298606839,"num_objects":72909,"num_object_clones":0,"num_object_copies":145818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31547,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213721,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9195,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":764768256,"data_stored":764694011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160743,"ondisk_log_size":160743,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640096,"kb_used_data":265484,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93731744,"statfs":{"total":96636764160,"available":95981305856,"internally_reserved":0,"allocated":271855616,"data_stored":271572290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":383577769},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593807,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567852,"kb_used_data":221008,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93803988,"statfs":{"total":96636764160,"available":96055283712,"internally_reserved":0,"allocated":226312192,"data_stored":225973202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,5,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637820,"kb_used_data":263276,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93734020,"statfs":{"total":96636764160,"available":95983636480,"internally_reserved":0,"allocated":269594624,"data_stored":269315370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":269090816,"data_stored":269087337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224608256,"data_stored":224545041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":271069184,"data_stored":271061633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:10.579 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:11.369 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:11.370 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:11.441 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-16T04:56:10.447965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":298606858,"num_objects":72911,"num_object_clones":0,"num_object_copies":145822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31547,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72911,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213723,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9196,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1845768,"kb_used_data":749768,"kb_used_omap":12,"kb_used_meta":1095923,"kb_avail":281269752,"statfs":{"total":289910292480,"available":288020226048,"internally_reserved":0,"allocated":767762432,"data_stored":766860862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1122225153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,2,4,10,11,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28889088,"num_objects":-7053,"num_object_clones":0,"num_object_copies":-14106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7053,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006720"},"pg_stats":[{"pgid":"2.f","version":"49'15295","reported_seq":15380,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.400626+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:09.400626+0000","last_peered":"2024-09-16T04:56:09.400626+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:09.400626+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18198528,"num_objects":4443,"num_object_clones":0,"num_object_copies":8886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15295,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1102}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15532","reported_seq":15654,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.031770+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:09.031770+0000","last_peered":"2024-09-16T04:56:09.031770+0000","last_clean":"2024-09-16T04:56:09.031770+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:09.031770+0000","last_undegraded":"2024-09-16T04:56:09.031770+0000","last_fullsized":"2024-09-16T04:56:09.031770+0000","mapping_epoch":35,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18276352,"num_objects":4462,"num_object_clones":0,"num_object_copies":8924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12501,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15334","reported_seq":15479,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:08.997430+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:08.997430+0000","last_peered":"2024-09-16T04:56:08.997430+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:08.997430+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18526232,"num_objects":4524,"num_object_clones":0,"num_object_copies":9048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12073,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15651","reported_seq":15727,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.030547+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:09.030547+0000","last_peered":"2024-09-16T04:56:09.030547+0000","last_clean":"2024-09-16T04:56:09.030547+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:09.030547+0000","last_undegraded":"2024-09-16T04:56:09.030547+0000","last_fullsized":"2024-09-16T04:56:09.030547+0000","mapping_epoch":35,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19222528,"num_objects":4693,"num_object_clones":0,"num_object_copies":9386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15651,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15572","reported_seq":15796,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.410015+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:09.410015+0000","last_peered":"2024-09-16T04:56:09.410015+0000","last_clean":"2024-09-16T04:56:09.410015+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:09.410015+0000","last_undegraded":"2024-09-16T04:56:09.410015+0000","last_fullsized":"2024-09-16T04:56:09.410015+0000","mapping_epoch":34,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18726912,"num_objects":4572,"num_object_clones":0,"num_object_copies":9144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12604,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15743","reported_seq":15882,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.380751+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:09.380751+0000","last_peered":"2024-09-16T04:56:09.380751+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:09.380751+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18718720,"num_objects":4570,"num_object_clones":0,"num_object_copies":9140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12736,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15374","reported_seq":15528,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.336089+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:09.336089+0000","last_peered":"2024-09-16T04:56:09.336089+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:09.336089+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18374656,"num_objects":4486,"num_object_clones":0,"num_object_copies":8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12029,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15435","reported_seq":15657,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.366746+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:09.366746+0000","last_peered":"2024-09-16T04:56:09.366746+0000","last_clean":"2024-09-16T04:56:09.366746+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:09.366746+0000","last_undegraded":"2024-09-16T04:56:09.366746+0000","last_fullsized":"2024-09-16T04:56:09.366746+0000","mapping_epoch":34,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18763776,"num_objects":4581,"num_object_clones":0,"num_object_copies":9162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12505,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15485","reported_seq":15719,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:08.952601+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:08.952601+0000","last_peered":"2024-09-16T04:56:08.952601+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:08.952601+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18825216,"num_objects":4596,"num_object_clones":0,"num_object_copies":9192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12236,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1275}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15701","reported_seq":15837,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.364389+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:09.364389+0000","last_peered":"2024-09-16T04:56:09.364389+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:09.364389+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18501632,"num_objects":4517,"num_object_clones":0,"num_object_copies":9034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12356,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1052}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15414","reported_seq":20688,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.016959+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:09.016959+0000","last_peered":"2024-09-16T04:56:09.016959+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:09.016959+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18699474,"num_objects":4567,"num_object_clones":0,"num_object_copies":9134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15386,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2515,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3782}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15481","reported_seq":22343,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.018885+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:09.018885+0000","last_peered":"2024-09-16T04:56:09.018885+0000","last_clean":"2024-09-16T04:56:09.018885+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:09.018885+0000","last_undegraded":"2024-09-16T04:56:09.018885+0000","last_fullsized":"2024-09-16T04:56:09.018885+0000","mapping_epoch":46,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18419712,"num_objects":4497,"num_object_clones":0,"num_object_copies":8994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12204,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15413","reported_seq":15515,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.017835+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:09.017835+0000","last_peered":"2024-09-16T04:56:09.017835+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:09.017835+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18281065,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15412,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1057}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15825","reported_seq":15934,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.002807+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:09.002807+0000","last_peered":"2024-09-16T04:56:09.002807+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:09.002807+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19281723,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15822,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1196}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15546","reported_seq":22345,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:08.542608+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:08.542608+0000","last_peered":"2024-09-16T04:56:08.542608+0000","last_clean":"2024-09-16T04:56:08.542608+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:08.542608+0000","last_undegraded":"2024-09-16T04:56:08.542608+0000","last_fullsized":"2024-09-16T04:56:08.542608+0000","mapping_epoch":48,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19054592,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12300,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15842","reported_seq":15988,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.389022+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:09.389022+0000","last_peered":"2024-09-16T04:56:09.389022+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:09.389022+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18735721,"num_objects":4575,"num_object_clones":0,"num_object_copies":9150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12611,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":298606839,"num_objects":72909,"num_object_clones":0,"num_object_copies":145818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31547,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213721,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9195,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":764768256,"data_stored":764694011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160743,"ondisk_log_size":160743,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640096,"kb_used_data":265484,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93731744,"statfs":{"total":96636764160,"available":95981305856,"internally_reserved":0,"allocated":271855616,"data_stored":271572290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":383577769},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593807,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567852,"kb_used_data":221008,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93803988,"statfs":{"total":96636764160,"available":96055283712,"internally_reserved":0,"allocated":226312192,"data_stored":225973202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,5,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637820,"kb_used_data":263276,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93734020,"statfs":{"total":96636764160,"available":95983636480,"internally_reserved":0,"allocated":269594624,"data_stored":269315370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":269090816,"data_stored":269087337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224608256,"data_stored":224545041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":271069184,"data_stored":271061633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:11.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:56:11.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'15295", 'reported_seq': 15380, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:09.400626+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:56:09.400626+0000', 'last_peered': '2024-09-16T04:56:09.400626+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:56:09.400626+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'5200", 'ondisk_log_start': "45'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18198528, 'num_objects': 4443, 'num_object_clones': 0, 'num_object_copies': 8886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4443, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15295, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1102}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:11.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:56:11.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'15334", 'reported_seq': 15479, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:08.997430+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:56:08.997430+0000', 'last_peered': '2024-09-16T04:56:08.997430+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:56:08.997430+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'5300", 'ondisk_log_start': "45'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18526232, 'num_objects': 4524, 'num_object_clones': 0, 'num_object_copies': 9048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4524, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12073, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:11.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:56:11.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'15743", 'reported_seq': 15882, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:09.380751+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:56:09.380751+0000', 'last_peered': '2024-09-16T04:56:09.380751+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:56:09.380751+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'5700", 'ondisk_log_start': "45'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18718720, 'num_objects': 4570, 'num_object_clones': 0, 'num_object_copies': 9140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12736, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:11.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:56:11.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'15374", 'reported_seq': 15528, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:09.336089+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:56:09.336089+0000', 'last_peered': '2024-09-16T04:56:09.336089+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:56:09.336089+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'5300", 'ondisk_log_start': "45'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18374656, 'num_objects': 4486, 'num_object_clones': 0, 'num_object_copies': 8972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4486, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12029, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1087}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:11.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:56:11.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'15485", 'reported_seq': 15719, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:08.952601+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:56:08.952601+0000', 'last_peered': '2024-09-16T04:56:08.952601+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:56:08.952601+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'5400", 'ondisk_log_start': "45'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18825216, 'num_objects': 4596, 'num_object_clones': 0, 'num_object_copies': 9192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4596, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12236, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1275}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:11.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:56:11.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'15701", 'reported_seq': 15837, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:09.364389+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:56:09.364389+0000', 'last_peered': '2024-09-16T04:56:09.364389+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:56:09.364389+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'5700", 'ondisk_log_start': "45'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18501632, 'num_objects': 4517, 'num_object_clones': 0, 'num_object_copies': 9034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4517, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12356, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1052}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:11.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:56:11.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'15414", 'reported_seq': 20688, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:09.016959+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:56:09.016959+0000', 'last_peered': '2024-09-16T04:56:09.016959+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:56:09.016959+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'5400", 'ondisk_log_start': "45'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18699474, 'num_objects': 4567, 'num_object_clones': 0, 'num_object_copies': 9134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 785, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4567, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15386, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2515, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3782}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:11.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:56:11.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'15413", 'reported_seq': 15515, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:09.017835+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:56:09.017835+0000', 'last_peered': '2024-09-16T04:56:09.017835+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:56:09.017835+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'5400", 'ondisk_log_start': "45'5400", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18281065, 'num_objects': 4464, 'num_object_clones': 0, 'num_object_copies': 8928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15412, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:11.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:56:11.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'15825", 'reported_seq': 15934, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:09.002807+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:56:09.002807+0000', 'last_peered': '2024-09-16T04:56:09.002807+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:56:09.002807+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'5800", 'ondisk_log_start': "45'5800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19281723, 'num_objects': 4710, 'num_object_clones': 0, 'num_object_copies': 9420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4710, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15822, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1196}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:11.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:56:11.444 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'15842", 'reported_seq': 15988, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:09.389022+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:56:09.389022+0000', 'last_peered': '2024-09-16T04:56:09.389022+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:56:09.389022+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'5800", 'ondisk_log_start': "45'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18735721, 'num_objects': 4575, 'num_object_clones': 0, 'num_object_copies': 9150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4575, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12611, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1051}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:11.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:56:12.463 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:12.464 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31547/145822 objects degraded (21.634%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":520,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":72911,"data_bytes":298606858,"bytes_used":1890066432,"bytes_avail":288020226048,"bytes_total":289910292480,"degraded_objects":31547,"degraded_total":145822,"degraded_ratio":0.21633909835278631,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":368,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":587},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:56:12.528 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:56:12.528 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:13.374 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:13.376 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:13.450 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-16T04:56:12.451729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":298606858,"num_objects":72911,"num_object_clones":0,"num_object_copies":145822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31547,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72911,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213723,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9196,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1845768,"kb_used_data":749768,"kb_used_omap":12,"kb_used_meta":1095923,"kb_avail":281269752,"statfs":{"total":289910292480,"available":288020226048,"internally_reserved":0,"allocated":767762432,"data_stored":766860862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1122225153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,2,4,10,11,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19304448,"num_objects":-4713,"num_object_clones":0,"num_object_copies":-9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008311"},"pg_stats":[{"pgid":"2.f","version":"49'15295","reported_seq":15380,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.400626+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:09.400626+0000","last_peered":"2024-09-16T04:56:09.400626+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:09.400626+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5200","ondisk_log_start":"45'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18198528,"num_objects":4443,"num_object_clones":0,"num_object_copies":8886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15295,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1102}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15532","reported_seq":15654,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.031770+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:09.031770+0000","last_peered":"2024-09-16T04:56:09.031770+0000","last_clean":"2024-09-16T04:56:09.031770+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:09.031770+0000","last_undegraded":"2024-09-16T04:56:09.031770+0000","last_fullsized":"2024-09-16T04:56:09.031770+0000","mapping_epoch":35,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18276352,"num_objects":4462,"num_object_clones":0,"num_object_copies":8924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12501,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15334","reported_seq":15479,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:08.997430+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:08.997430+0000","last_peered":"2024-09-16T04:56:08.997430+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:08.997430+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18526232,"num_objects":4524,"num_object_clones":0,"num_object_copies":9048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12073,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15651","reported_seq":15727,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.030547+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:09.030547+0000","last_peered":"2024-09-16T04:56:09.030547+0000","last_clean":"2024-09-16T04:56:09.030547+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:09.030547+0000","last_undegraded":"2024-09-16T04:56:09.030547+0000","last_fullsized":"2024-09-16T04:56:09.030547+0000","mapping_epoch":35,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19222528,"num_objects":4693,"num_object_clones":0,"num_object_copies":9386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15651,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15572","reported_seq":15796,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.410015+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:09.410015+0000","last_peered":"2024-09-16T04:56:09.410015+0000","last_clean":"2024-09-16T04:56:09.410015+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:09.410015+0000","last_undegraded":"2024-09-16T04:56:09.410015+0000","last_fullsized":"2024-09-16T04:56:09.410015+0000","mapping_epoch":34,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18726912,"num_objects":4572,"num_object_clones":0,"num_object_copies":9144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12604,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15743","reported_seq":15882,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.380751+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:09.380751+0000","last_peered":"2024-09-16T04:56:09.380751+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:09.380751+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18718720,"num_objects":4570,"num_object_clones":0,"num_object_copies":9140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12736,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15374","reported_seq":15528,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.336089+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:09.336089+0000","last_peered":"2024-09-16T04:56:09.336089+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:09.336089+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5300","ondisk_log_start":"45'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18374656,"num_objects":4486,"num_object_clones":0,"num_object_copies":8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12029,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15435","reported_seq":15657,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.366746+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:09.366746+0000","last_peered":"2024-09-16T04:56:09.366746+0000","last_clean":"2024-09-16T04:56:09.366746+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:09.366746+0000","last_undegraded":"2024-09-16T04:56:09.366746+0000","last_fullsized":"2024-09-16T04:56:09.366746+0000","mapping_epoch":34,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18763776,"num_objects":4581,"num_object_clones":0,"num_object_copies":9162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12505,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15485","reported_seq":15719,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:08.952601+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:08.952601+0000","last_peered":"2024-09-16T04:56:08.952601+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:08.952601+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18825216,"num_objects":4596,"num_object_clones":0,"num_object_copies":9192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12236,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1275}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15701","reported_seq":15837,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.364389+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:09.364389+0000","last_peered":"2024-09-16T04:56:09.364389+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:09.364389+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18501632,"num_objects":4517,"num_object_clones":0,"num_object_copies":9034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12356,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1052}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15414","reported_seq":20688,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.016959+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:09.016959+0000","last_peered":"2024-09-16T04:56:09.016959+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:09.016959+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18699474,"num_objects":4567,"num_object_clones":0,"num_object_copies":9134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15386,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2515,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3782}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15481","reported_seq":22343,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:09.018885+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:09.018885+0000","last_peered":"2024-09-16T04:56:09.018885+0000","last_clean":"2024-09-16T04:56:09.018885+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:09.018885+0000","last_undegraded":"2024-09-16T04:56:09.018885+0000","last_fullsized":"2024-09-16T04:56:09.018885+0000","mapping_epoch":46,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18419712,"num_objects":4497,"num_object_clones":0,"num_object_copies":8994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12204,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15413","reported_seq":15515,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.017835+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:09.017835+0000","last_peered":"2024-09-16T04:56:09.017835+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:09.017835+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18281065,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15412,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1057}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15825","reported_seq":15934,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.002807+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:09.002807+0000","last_peered":"2024-09-16T04:56:09.002807+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:09.002807+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19281723,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15822,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1196}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15546","reported_seq":22345,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:08.542608+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:08.542608+0000","last_peered":"2024-09-16T04:56:08.542608+0000","last_clean":"2024-09-16T04:56:08.542608+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:08.542608+0000","last_undegraded":"2024-09-16T04:56:08.542608+0000","last_fullsized":"2024-09-16T04:56:08.542608+0000","mapping_epoch":48,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19054592,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12300,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15842","reported_seq":15988,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:09.389022+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:09.389022+0000","last_peered":"2024-09-16T04:56:09.389022+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:09.389022+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18735721,"num_objects":4575,"num_object_clones":0,"num_object_copies":9150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12611,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1051}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":298606839,"num_objects":72909,"num_object_clones":0,"num_object_copies":145818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31547,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213721,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9195,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":764768256,"data_stored":764694011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160743,"ondisk_log_size":160743,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640096,"kb_used_data":265484,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93731744,"statfs":{"total":96636764160,"available":95981305856,"internally_reserved":0,"allocated":271855616,"data_stored":271572290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":383577769},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593807,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567852,"kb_used_data":221008,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93803988,"statfs":{"total":96636764160,"available":96055283712,"internally_reserved":0,"allocated":226312192,"data_stored":225973202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,5,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637820,"kb_used_data":263276,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93734020,"statfs":{"total":96636764160,"available":95983636480,"internally_reserved":0,"allocated":269594624,"data_stored":269315370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":269090816,"data_stored":269087337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224608256,"data_stored":224545041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":271069184,"data_stored":271061633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:16.452 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:17.293 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:17.295 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:17.331 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-16T04:56:16.452618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":289141002,"num_objects":70600,"num_object_clones":0,"num_object_copies":141200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216034,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9314,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160757,"ondisk_log_size":160757,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1831356,"kb_used_data":731196,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281284164,"statfs":{"total":289910292480,"available":288034983936,"internally_reserved":0,"allocated":748744704,"data_stored":747843134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1126484993},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,10,9,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19128320,"num_objects":-4670,"num_object_clones":0,"num_object_copies":-9340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4670,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007567"},"pg_stats":[{"pgid":"2.f","version":"49'15440","reported_seq":15525,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.259145+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:14.259145+0000","last_peered":"2024-09-16T04:56:14.259145+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:14.259145+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17604608,"num_objects":4298,"num_object_clones":0,"num_object_copies":8596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15440,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":957}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15680","reported_seq":15802,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:13.935220+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:13.935220+0000","last_peered":"2024-09-16T04:56:13.935220+0000","last_clean":"2024-09-16T04:56:13.935220+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:13.935220+0000","last_undegraded":"2024-09-16T04:56:13.935220+0000","last_fullsized":"2024-09-16T04:56:13.935220+0000","mapping_epoch":35,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17670144,"num_objects":4314,"num_object_clones":0,"num_object_copies":8628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12649,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15485","reported_seq":15630,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:13.999697+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:13.999697+0000","last_peered":"2024-09-16T04:56:13.999697+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:13.999697+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907736,"num_objects":4373,"num_object_clones":0,"num_object_copies":8746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12224,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1054}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15809","reported_seq":15885,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:14.015465+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:14.015465+0000","last_peered":"2024-09-16T04:56:14.015465+0000","last_clean":"2024-09-16T04:56:14.015465+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:14.015465+0000","last_undegraded":"2024-09-16T04:56:14.015465+0000","last_fullsized":"2024-09-16T04:56:14.015465+0000","mapping_epoch":35,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18575360,"num_objects":4535,"num_object_clones":0,"num_object_copies":9070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15809,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15720","reported_seq":15945,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:14.345433+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:14.345433+0000","last_peered":"2024-09-16T04:56:14.345433+0000","last_clean":"2024-09-16T04:56:14.345433+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:14.345433+0000","last_undegraded":"2024-09-16T04:56:14.345433+0000","last_fullsized":"2024-09-16T04:56:14.345433+0000","mapping_epoch":34,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18120704,"num_objects":4424,"num_object_clones":0,"num_object_copies":8848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12752,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15884","reported_seq":16020,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.424070+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:14.424070+0000","last_peered":"2024-09-16T04:56:14.424070+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:14.424070+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18141184,"num_objects":4429,"num_object_clones":0,"num_object_copies":8858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12877,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":973}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15517","reported_seq":15671,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.430598+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:14.430598+0000","last_peered":"2024-09-16T04:56:14.430598+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:14.430598+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17788928,"num_objects":4343,"num_object_clones":0,"num_object_copies":8686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12172,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15577","reported_seq":15799,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:14.390515+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:14.390515+0000","last_peered":"2024-09-16T04:56:14.390515+0000","last_clean":"2024-09-16T04:56:14.390515+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:14.390515+0000","last_undegraded":"2024-09-16T04:56:14.390515+0000","last_fullsized":"2024-09-16T04:56:14.390515+0000","mapping_epoch":34,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18182144,"num_objects":4439,"num_object_clones":0,"num_object_copies":8878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12647,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15628","reported_seq":15862,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:13.967412+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:13.967412+0000","last_peered":"2024-09-16T04:56:13.967412+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:13.967412+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18239488,"num_objects":4453,"num_object_clones":0,"num_object_copies":8906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12379,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1132}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15836","reported_seq":15972,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.440177+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:14.440177+0000","last_peered":"2024-09-16T04:56:14.440177+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:14.440177+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17948672,"num_objects":4382,"num_object_clones":0,"num_object_copies":8764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12491,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":917}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15562","reported_seq":21071,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.055103+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:14.055103+0000","last_peered":"2024-09-16T04:56:14.055103+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:14.055103+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18093266,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15534,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3752}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15617","reported_seq":22479,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:13.975321+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:13.975321+0000","last_peered":"2024-09-16T04:56:13.975321+0000","last_clean":"2024-09-16T04:56:13.975321+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:13.975321+0000","last_undegraded":"2024-09-16T04:56:13.975321+0000","last_fullsized":"2024-09-16T04:56:13.975321+0000","mapping_epoch":46,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17862656,"num_objects":4361,"num_object_clones":0,"num_object_copies":8722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12340,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15548","reported_seq":15650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.034284+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:14.034284+0000","last_peered":"2024-09-16T04:56:14.034284+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:14.034284+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17728105,"num_objects":4329,"num_object_clones":0,"num_object_copies":8658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15547,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15968","reported_seq":16076,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.021871+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:14.021871+0000","last_peered":"2024-09-16T04:56:14.021871+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:14.021871+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18695995,"num_objects":4567,"num_object_clones":0,"num_object_copies":9134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15965,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1053}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15708","reported_seq":22507,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:13.624328+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:13.624328+0000","last_peered":"2024-09-16T04:56:13.624328+0000","last_clean":"2024-09-16T04:56:13.624328+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:13.624328+0000","last_undegraded":"2024-09-16T04:56:13.624328+0000","last_fullsized":"2024-09-16T04:56:13.624328+0000","mapping_epoch":48,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18391040,"num_objects":4490,"num_object_clones":0,"num_object_copies":8980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12462,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15975","reported_seq":16121,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.401719+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:14.401719+0000","last_peered":"2024-09-16T04:56:14.401719+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:14.401719+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18190953,"num_objects":4442,"num_object_clones":0,"num_object_copies":8884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12744,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":289140983,"num_objects":70598,"num_object_clones":0,"num_object_copies":141196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216032,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9313,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":745717760,"data_stored":745643515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642776,"kb_used_data":259844,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93729064,"statfs":{"total":96636764160,"available":95978561536,"internally_reserved":0,"allocated":266080256,"data_stored":265796930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":392097449},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,6,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593808,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552444,"kb_used_data":213920,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93819396,"statfs":{"total":96636764160,"available":96071061504,"internally_reserved":0,"allocated":219054080,"data_stored":218715090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636136,"kb_used_data":257432,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93735704,"statfs":{"total":96636764160,"available":95985360896,"internally_reserved":0,"allocated":263610368,"data_stored":263331114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":263147520,"data_stored":263144041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":217444352,"data_stored":217381137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":265125888,"data_stored":265118337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:17.332 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:18.121 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:18.123 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:18.205 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-16T04:56:16.452618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":289141002,"num_objects":70600,"num_object_clones":0,"num_object_copies":141200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216034,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9314,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160757,"ondisk_log_size":160757,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1831356,"kb_used_data":731196,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281284164,"statfs":{"total":289910292480,"available":288034983936,"internally_reserved":0,"allocated":748744704,"data_stored":747843134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1126484993},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,10,9,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19128320,"num_objects":-4670,"num_object_clones":0,"num_object_copies":-9340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4670,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007567"},"pg_stats":[{"pgid":"2.f","version":"49'15440","reported_seq":15525,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.259145+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:14.259145+0000","last_peered":"2024-09-16T04:56:14.259145+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:14.259145+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17604608,"num_objects":4298,"num_object_clones":0,"num_object_copies":8596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15440,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":957}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15680","reported_seq":15802,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:13.935220+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:13.935220+0000","last_peered":"2024-09-16T04:56:13.935220+0000","last_clean":"2024-09-16T04:56:13.935220+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:13.935220+0000","last_undegraded":"2024-09-16T04:56:13.935220+0000","last_fullsized":"2024-09-16T04:56:13.935220+0000","mapping_epoch":35,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17670144,"num_objects":4314,"num_object_clones":0,"num_object_copies":8628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12649,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15485","reported_seq":15630,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:13.999697+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:13.999697+0000","last_peered":"2024-09-16T04:56:13.999697+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:13.999697+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907736,"num_objects":4373,"num_object_clones":0,"num_object_copies":8746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12224,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1054}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15809","reported_seq":15885,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:14.015465+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:14.015465+0000","last_peered":"2024-09-16T04:56:14.015465+0000","last_clean":"2024-09-16T04:56:14.015465+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:14.015465+0000","last_undegraded":"2024-09-16T04:56:14.015465+0000","last_fullsized":"2024-09-16T04:56:14.015465+0000","mapping_epoch":35,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18575360,"num_objects":4535,"num_object_clones":0,"num_object_copies":9070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15809,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15720","reported_seq":15945,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:14.345433+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:14.345433+0000","last_peered":"2024-09-16T04:56:14.345433+0000","last_clean":"2024-09-16T04:56:14.345433+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:14.345433+0000","last_undegraded":"2024-09-16T04:56:14.345433+0000","last_fullsized":"2024-09-16T04:56:14.345433+0000","mapping_epoch":34,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18120704,"num_objects":4424,"num_object_clones":0,"num_object_copies":8848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12752,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15884","reported_seq":16020,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.424070+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:14.424070+0000","last_peered":"2024-09-16T04:56:14.424070+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:14.424070+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18141184,"num_objects":4429,"num_object_clones":0,"num_object_copies":8858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12877,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":973}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15517","reported_seq":15671,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.430598+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:14.430598+0000","last_peered":"2024-09-16T04:56:14.430598+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:14.430598+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17788928,"num_objects":4343,"num_object_clones":0,"num_object_copies":8686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12172,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15577","reported_seq":15799,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:14.390515+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:14.390515+0000","last_peered":"2024-09-16T04:56:14.390515+0000","last_clean":"2024-09-16T04:56:14.390515+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:14.390515+0000","last_undegraded":"2024-09-16T04:56:14.390515+0000","last_fullsized":"2024-09-16T04:56:14.390515+0000","mapping_epoch":34,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18182144,"num_objects":4439,"num_object_clones":0,"num_object_copies":8878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12647,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15628","reported_seq":15862,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:13.967412+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:13.967412+0000","last_peered":"2024-09-16T04:56:13.967412+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:13.967412+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18239488,"num_objects":4453,"num_object_clones":0,"num_object_copies":8906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12379,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1132}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15836","reported_seq":15972,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.440177+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:14.440177+0000","last_peered":"2024-09-16T04:56:14.440177+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:14.440177+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17948672,"num_objects":4382,"num_object_clones":0,"num_object_copies":8764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12491,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":917}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15562","reported_seq":21071,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.055103+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:14.055103+0000","last_peered":"2024-09-16T04:56:14.055103+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:14.055103+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18093266,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15534,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3752}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15617","reported_seq":22479,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:13.975321+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:13.975321+0000","last_peered":"2024-09-16T04:56:13.975321+0000","last_clean":"2024-09-16T04:56:13.975321+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:13.975321+0000","last_undegraded":"2024-09-16T04:56:13.975321+0000","last_fullsized":"2024-09-16T04:56:13.975321+0000","mapping_epoch":46,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17862656,"num_objects":4361,"num_object_clones":0,"num_object_copies":8722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12340,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15548","reported_seq":15650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.034284+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:14.034284+0000","last_peered":"2024-09-16T04:56:14.034284+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:14.034284+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17728105,"num_objects":4329,"num_object_clones":0,"num_object_copies":8658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15547,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15968","reported_seq":16076,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.021871+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:14.021871+0000","last_peered":"2024-09-16T04:56:14.021871+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:14.021871+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18695995,"num_objects":4567,"num_object_clones":0,"num_object_copies":9134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15965,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1053}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15708","reported_seq":22507,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:13.624328+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:13.624328+0000","last_peered":"2024-09-16T04:56:13.624328+0000","last_clean":"2024-09-16T04:56:13.624328+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:13.624328+0000","last_undegraded":"2024-09-16T04:56:13.624328+0000","last_fullsized":"2024-09-16T04:56:13.624328+0000","mapping_epoch":48,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18391040,"num_objects":4490,"num_object_clones":0,"num_object_copies":8980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12462,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15975","reported_seq":16121,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.401719+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:14.401719+0000","last_peered":"2024-09-16T04:56:14.401719+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:14.401719+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18190953,"num_objects":4442,"num_object_clones":0,"num_object_copies":8884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12744,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":289140983,"num_objects":70598,"num_object_clones":0,"num_object_copies":141196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216032,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9313,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":745717760,"data_stored":745643515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642776,"kb_used_data":259844,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93729064,"statfs":{"total":96636764160,"available":95978561536,"internally_reserved":0,"allocated":266080256,"data_stored":265796930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":392097449},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,6,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593808,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552444,"kb_used_data":213920,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93819396,"statfs":{"total":96636764160,"available":96071061504,"internally_reserved":0,"allocated":219054080,"data_stored":218715090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636136,"kb_used_data":257432,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93735704,"statfs":{"total":96636764160,"available":95985360896,"internally_reserved":0,"allocated":263610368,"data_stored":263331114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":263147520,"data_stored":263144041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":217444352,"data_stored":217381137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":265125888,"data_stored":265118337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:18.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:56:18.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'15440", 'reported_seq': 15525, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:14.259145+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:56:14.259145+0000', 'last_peered': '2024-09-16T04:56:14.259145+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:56:14.259145+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'5400", 'ondisk_log_start': "45'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17604608, 'num_objects': 4298, 'num_object_clones': 0, 'num_object_copies': 8596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4298, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15440, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 957}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:18.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:56:18.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'15485", 'reported_seq': 15630, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:13.999697+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:56:13.999697+0000', 'last_peered': '2024-09-16T04:56:13.999697+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:56:13.999697+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'5400", 'ondisk_log_start': "45'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17907736, 'num_objects': 4373, 'num_object_clones': 0, 'num_object_copies': 8746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4373, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12224, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1054}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:18.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:56:18.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'15884", 'reported_seq': 16020, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:14.424070+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:56:14.424070+0000', 'last_peered': '2024-09-16T04:56:14.424070+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:56:14.424070+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'5800", 'ondisk_log_start': "45'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18141184, 'num_objects': 4429, 'num_object_clones': 0, 'num_object_copies': 8858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12877, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 973}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:18.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:56:18.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'15517", 'reported_seq': 15671, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:14.430598+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:56:14.430598+0000', 'last_peered': '2024-09-16T04:56:14.430598+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:56:14.430598+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'5500", 'ondisk_log_start': "45'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17788928, 'num_objects': 4343, 'num_object_clones': 0, 'num_object_copies': 8686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4343, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12172, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 944}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:18.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:56:18.207 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'15628", 'reported_seq': 15862, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:13.967412+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:56:13.967412+0000', 'last_peered': '2024-09-16T04:56:13.967412+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:56:13.967412+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'5600", 'ondisk_log_start': "45'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18239488, 'num_objects': 4453, 'num_object_clones': 0, 'num_object_copies': 8906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4453, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12379, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1132}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:18.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:56:18.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'15836", 'reported_seq': 15972, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:14.440177+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:56:14.440177+0000', 'last_peered': '2024-09-16T04:56:14.440177+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:56:14.440177+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'5800", 'ondisk_log_start': "45'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17948672, 'num_objects': 4382, 'num_object_clones': 0, 'num_object_copies': 8764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4382, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12491, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 917}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:18.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:56:18.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'15562", 'reported_seq': 21071, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:14.055103+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:56:14.055103+0000', 'last_peered': '2024-09-16T04:56:14.055103+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:56:14.055103+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'5500", 'ondisk_log_start': "45'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18093266, 'num_objects': 4419, 'num_object_clones': 0, 'num_object_copies': 8838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 667, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4419, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15534, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2633, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3752}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:18.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:56:18.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'15548", 'reported_seq': 15650, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:14.034284+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:56:14.034284+0000', 'last_peered': '2024-09-16T04:56:14.034284+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:56:14.034284+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'5500", 'ondisk_log_start': "45'5500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17728105, 'num_objects': 4329, 'num_object_clones': 0, 'num_object_copies': 8658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4329, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15547, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 922}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:18.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:56:18.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'15968", 'reported_seq': 16076, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:14.021871+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:56:14.021871+0000', 'last_peered': '2024-09-16T04:56:14.021871+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:56:14.021871+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'5900", 'ondisk_log_start': "45'5900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18695995, 'num_objects': 4567, 'num_object_clones': 0, 'num_object_copies': 9134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4567, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15965, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1053}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:18.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:56:18.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'15975", 'reported_seq': 16121, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:14.401719+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:56:14.401719+0000', 'last_peered': '2024-09-16T04:56:14.401719+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:56:14.401719+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'5900", 'ondisk_log_start': "45'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18190953, 'num_objects': 4442, 'num_object_clones': 0, 'num_object_copies': 8884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12744, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 918}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:18.209 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:56:19.280 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:19.280 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31429/141200 objects degraded (22.258%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":527,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":70600,"data_bytes":289141002,"bytes_used":1875308544,"bytes_avail":288034983936,"bytes_total":289910292480,"degraded_objects":31429,"degraded_total":141200,"degraded_ratio":0.22258498583569405,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":388},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:56:19.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:56:19.338 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:20.002 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:20.006 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:20.057 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-16T04:56:18.454450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":289141002,"num_objects":70600,"num_object_clones":0,"num_object_copies":141200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216034,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9314,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160757,"ondisk_log_size":160757,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1831356,"kb_used_data":731196,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281284164,"statfs":{"total":289910292480,"available":288034983936,"internally_reserved":0,"allocated":748744704,"data_stored":747843134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1126484993},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,10,9,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19128320,"num_objects":-4670,"num_object_clones":0,"num_object_copies":-9340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4670,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008924"},"pg_stats":[{"pgid":"2.f","version":"49'15440","reported_seq":15525,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.259145+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:14.259145+0000","last_peered":"2024-09-16T04:56:14.259145+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:14.259145+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17604608,"num_objects":4298,"num_object_clones":0,"num_object_copies":8596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15440,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":957}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15680","reported_seq":15802,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:13.935220+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:13.935220+0000","last_peered":"2024-09-16T04:56:13.935220+0000","last_clean":"2024-09-16T04:56:13.935220+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:13.935220+0000","last_undegraded":"2024-09-16T04:56:13.935220+0000","last_fullsized":"2024-09-16T04:56:13.935220+0000","mapping_epoch":35,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17670144,"num_objects":4314,"num_object_clones":0,"num_object_copies":8628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12649,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15485","reported_seq":15630,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:13.999697+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:13.999697+0000","last_peered":"2024-09-16T04:56:13.999697+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:13.999697+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5400","ondisk_log_start":"45'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17907736,"num_objects":4373,"num_object_clones":0,"num_object_copies":8746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12224,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1054}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15809","reported_seq":15885,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:14.015465+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:14.015465+0000","last_peered":"2024-09-16T04:56:14.015465+0000","last_clean":"2024-09-16T04:56:14.015465+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:14.015465+0000","last_undegraded":"2024-09-16T04:56:14.015465+0000","last_fullsized":"2024-09-16T04:56:14.015465+0000","mapping_epoch":35,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18575360,"num_objects":4535,"num_object_clones":0,"num_object_copies":9070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15809,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15720","reported_seq":15945,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:14.345433+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:14.345433+0000","last_peered":"2024-09-16T04:56:14.345433+0000","last_clean":"2024-09-16T04:56:14.345433+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:14.345433+0000","last_undegraded":"2024-09-16T04:56:14.345433+0000","last_fullsized":"2024-09-16T04:56:14.345433+0000","mapping_epoch":34,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18120704,"num_objects":4424,"num_object_clones":0,"num_object_copies":8848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12752,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'15884","reported_seq":16020,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.424070+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:14.424070+0000","last_peered":"2024-09-16T04:56:14.424070+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:14.424070+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18141184,"num_objects":4429,"num_object_clones":0,"num_object_copies":8858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12877,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":973}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15517","reported_seq":15671,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.430598+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:14.430598+0000","last_peered":"2024-09-16T04:56:14.430598+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:14.430598+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17788928,"num_objects":4343,"num_object_clones":0,"num_object_copies":8686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12172,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15577","reported_seq":15799,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:14.390515+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:14.390515+0000","last_peered":"2024-09-16T04:56:14.390515+0000","last_clean":"2024-09-16T04:56:14.390515+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:14.390515+0000","last_undegraded":"2024-09-16T04:56:14.390515+0000","last_fullsized":"2024-09-16T04:56:14.390515+0000","mapping_epoch":34,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18182144,"num_objects":4439,"num_object_clones":0,"num_object_copies":8878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12647,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15628","reported_seq":15862,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:13.967412+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:13.967412+0000","last_peered":"2024-09-16T04:56:13.967412+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:13.967412+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18239488,"num_objects":4453,"num_object_clones":0,"num_object_copies":8906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12379,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1132}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'15836","reported_seq":15972,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.440177+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:14.440177+0000","last_peered":"2024-09-16T04:56:14.440177+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:14.440177+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17948672,"num_objects":4382,"num_object_clones":0,"num_object_copies":8764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12491,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":917}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15562","reported_seq":21071,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.055103+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:14.055103+0000","last_peered":"2024-09-16T04:56:14.055103+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:14.055103+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18093266,"num_objects":4419,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15534,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2633,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3752}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15617","reported_seq":22479,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:13.975321+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:13.975321+0000","last_peered":"2024-09-16T04:56:13.975321+0000","last_clean":"2024-09-16T04:56:13.975321+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:13.975321+0000","last_undegraded":"2024-09-16T04:56:13.975321+0000","last_fullsized":"2024-09-16T04:56:13.975321+0000","mapping_epoch":46,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17862656,"num_objects":4361,"num_object_clones":0,"num_object_copies":8722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12340,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15548","reported_seq":15650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.034284+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:14.034284+0000","last_peered":"2024-09-16T04:56:14.034284+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:14.034284+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17728105,"num_objects":4329,"num_object_clones":0,"num_object_copies":8658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15547,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":922}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'15968","reported_seq":16076,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.021871+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:14.021871+0000","last_peered":"2024-09-16T04:56:14.021871+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:14.021871+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18695995,"num_objects":4567,"num_object_clones":0,"num_object_copies":9134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15965,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1053}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15708","reported_seq":22507,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:13.624328+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:13.624328+0000","last_peered":"2024-09-16T04:56:13.624328+0000","last_clean":"2024-09-16T04:56:13.624328+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:13.624328+0000","last_undegraded":"2024-09-16T04:56:13.624328+0000","last_fullsized":"2024-09-16T04:56:13.624328+0000","mapping_epoch":48,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18391040,"num_objects":4490,"num_object_clones":0,"num_object_copies":8980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12462,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'15975","reported_seq":16121,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:14.401719+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:14.401719+0000","last_peered":"2024-09-16T04:56:14.401719+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:14.401719+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18190953,"num_objects":4442,"num_object_clones":0,"num_object_copies":8884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12744,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":289140983,"num_objects":70598,"num_object_clones":0,"num_object_copies":141196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70598,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216032,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9313,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":745717760,"data_stored":745643515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642776,"kb_used_data":259844,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93729064,"statfs":{"total":96636764160,"available":95978561536,"internally_reserved":0,"allocated":266080256,"data_stored":265796930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":392097449},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,6,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593808,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552444,"kb_used_data":213920,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93819396,"statfs":{"total":96636764160,"available":96071061504,"internally_reserved":0,"allocated":219054080,"data_stored":218715090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636136,"kb_used_data":257432,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93735704,"statfs":{"total":96636764160,"available":95985360896,"internally_reserved":0,"allocated":263610368,"data_stored":263331114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":263147520,"data_stored":263144041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":217444352,"data_stored":217381137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":265125888,"data_stored":265118337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:23.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:23.746 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:23.746 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:23.786 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-09-16T04:56:22.455376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":279494922,"num_objects":68245,"num_object_clones":0,"num_object_copies":136490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31309,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218389,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9434,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160712,"ondisk_log_size":160712,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1808128,"kb_used_data":712140,"kb_used_omap":12,"kb_used_meta":1095923,"kb_avail":281307392,"statfs":{"total":289910292480,"available":288058769408,"internally_reserved":0,"allocated":729231360,"data_stored":728329790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1122225153},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,7,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19111936,"num_objects":-4666,"num_object_clones":0,"num_object_copies":-9332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4666,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007411"},"pg_stats":[{"pgid":"2.f","version":"49'15585","reported_seq":15670,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:19.413074+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:19.413074+0000","last_peered":"2024-09-16T04:56:19.413074+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:19.413074+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17010688,"num_objects":4153,"num_object_clones":0,"num_object_copies":8306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15585,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":812}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15816","reported_seq":15938,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:19.092200+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:19.092200+0000","last_peered":"2024-09-16T04:56:19.092200+0000","last_clean":"2024-09-16T04:56:19.092200+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:19.092200+0000","last_undegraded":"2024-09-16T04:56:19.092200+0000","last_fullsized":"2024-09-16T04:56:19.092200+0000","mapping_epoch":35,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17113088,"num_objects":4178,"num_object_clones":0,"num_object_copies":8356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12785,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15640","reported_seq":15785,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:18.973397+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:18.973397+0000","last_peered":"2024-09-16T04:56:18.973397+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:18.973397+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17272856,"num_objects":4218,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4218,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12379,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":899}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'15964","reported_seq":16040,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:18.937727+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:18.937727+0000","last_peered":"2024-09-16T04:56:18.937727+0000","last_clean":"2024-09-16T04:56:18.937727+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:18.937727+0000","last_undegraded":"2024-09-16T04:56:18.937727+0000","last_fullsized":"2024-09-16T04:56:18.937727+0000","mapping_epoch":35,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17940480,"num_objects":4380,"num_object_clones":0,"num_object_copies":8760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15964,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15874","reported_seq":16099,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:19.448611+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:19.448611+0000","last_peered":"2024-09-16T04:56:19.448611+0000","last_clean":"2024-09-16T04:56:19.448611+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:19.448611+0000","last_undegraded":"2024-09-16T04:56:19.448611+0000","last_fullsized":"2024-09-16T04:56:19.448611+0000","mapping_epoch":34,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17489920,"num_objects":4270,"num_object_clones":0,"num_object_copies":8540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12906,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16022","reported_seq":16161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:19.440469+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:19.440469+0000","last_peered":"2024-09-16T04:56:19.440469+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:19.440469+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17575936,"num_objects":4291,"num_object_clones":0,"num_object_copies":8582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13015,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15661","reported_seq":15815,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:19.362009+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:19.362009+0000","last_peered":"2024-09-16T04:56:19.362009+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:19.362009+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17199104,"num_objects":4199,"num_object_clones":0,"num_object_copies":8398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12316,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":800}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15718","reported_seq":15940,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:19.380984+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:19.380984+0000","last_peered":"2024-09-16T04:56:19.380984+0000","last_clean":"2024-09-16T04:56:19.380984+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:19.380984+0000","last_undegraded":"2024-09-16T04:56:19.380984+0000","last_fullsized":"2024-09-16T04:56:19.380984+0000","mapping_epoch":34,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17604608,"num_objects":4298,"num_object_clones":0,"num_object_copies":8596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12788,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15767","reported_seq":16001,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:19.065126+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:19.065126+0000","last_peered":"2024-09-16T04:56:19.065126+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:19.065126+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17670144,"num_objects":4314,"num_object_clones":0,"num_object_copies":8628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12518,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":993}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16003","reported_seq":16134,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:19.381098+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:19.381098+0000","last_peered":"2024-09-16T04:56:19.381098+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:19.381098+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17264640,"num_objects":4215,"num_object_clones":0,"num_object_copies":8430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":750}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15716","reported_seq":21465,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:19.109381+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:19.109381+0000","last_peered":"2024-09-16T04:56:19.109381+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:19.109381+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17462482,"num_objects":4265,"num_object_clones":0,"num_object_copies":8530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":547,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15688,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2753,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3718}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15745","reported_seq":22607,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:19.098345+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:19.098345+0000","last_peered":"2024-09-16T04:56:19.098345+0000","last_clean":"2024-09-16T04:56:19.098345+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:19.098345+0000","last_undegraded":"2024-09-16T04:56:19.098345+0000","last_fullsized":"2024-09-16T04:56:19.098345+0000","mapping_epoch":46,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17338368,"num_objects":4233,"num_object_clones":0,"num_object_copies":8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12468,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15695","reported_seq":15798,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:19.103878+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:19.103878+0000","last_peered":"2024-09-16T04:56:19.103878+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:19.103878+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17125993,"num_objects":4182,"num_object_clones":0,"num_object_copies":8364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15694,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":775}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16112","reported_seq":16221,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:18.976975+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:18.976975+0000","last_peered":"2024-09-16T04:56:18.976975+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:18.976975+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18106171,"num_objects":4423,"num_object_clones":0,"num_object_copies":8846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16109,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":909}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'15856","reported_seq":22656,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:18.565408+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:18.565408+0000","last_peered":"2024-09-16T04:56:18.565408+0000","last_clean":"2024-09-16T04:56:18.565408+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:18.565408+0000","last_undegraded":"2024-09-16T04:56:18.565408+0000","last_fullsized":"2024-09-16T04:56:18.565408+0000","mapping_epoch":48,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17784832,"num_objects":4342,"num_object_clones":0,"num_object_copies":8684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12610,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16135","reported_seq":16281,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:19.428467+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:19.428467+0000","last_peered":"2024-09-16T04:56:19.428467+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:19.428467+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17535593,"num_objects":4282,"num_object_clones":0,"num_object_copies":8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12904,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":279494903,"num_objects":68243,"num_object_clones":0,"num_object_copies":136486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31309,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218387,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9433,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":726409216,"data_stored":726334971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160709,"ondisk_log_size":160709,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624448,"kb_used_data":254000,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93747392,"statfs":{"total":96636764160,"available":95997329408,"internally_reserved":0,"allocated":260096000,"data_stored":259812674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":379317929},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593809,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549368,"kb_used_data":206696,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93822472,"statfs":{"total":96636764160,"available":96074211328,"internally_reserved":0,"allocated":211656704,"data_stored":211317714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,5,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634312,"kb_used_data":251444,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93737528,"statfs":{"total":96636764160,"available":95987228672,"internally_reserved":0,"allocated":257478656,"data_stored":257199402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257052672,"data_stored":257049193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":210173952,"data_stored":210110737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":259182592,"data_stored":259175041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:23.788 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:24.640 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:24.641 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:24.706 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-16T04:56:24.455772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":273961226,"num_objects":66894,"num_object_clones":0,"num_object_copies":133788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66894,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219740,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9544,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160663,"ondisk_log_size":160663,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1809032,"kb_used_data":699152,"kb_used_omap":12,"kb_used_meta":1109811,"kb_avail":281306488,"statfs":{"total":289910292480,"available":288057843712,"internally_reserved":0,"allocated":715931648,"data_stored":715030078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1136446465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,7,13,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24645632,"num_objects":-6017,"num_object_clones":0,"num_object_copies":-12034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-348,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6017,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004042"},"pg_stats":[{"pgid":"2.f","version":"49'15585","reported_seq":15670,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:19.413074+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:19.413074+0000","last_peered":"2024-09-16T04:56:19.413074+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:19.413074+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5500","ondisk_log_start":"45'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17010688,"num_objects":4153,"num_object_clones":0,"num_object_copies":8306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15585,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":812}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15955","reported_seq":16075,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:24.071917+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:24.071917+0000","last_peered":"2024-09-16T04:56:24.071917+0000","last_clean":"2024-09-16T04:56:24.071917+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:24.071917+0000","last_undegraded":"2024-09-16T04:56:24.071917+0000","last_fullsized":"2024-09-16T04:56:24.071917+0000","mapping_epoch":35,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16543744,"num_objects":4039,"num_object_clones":0,"num_object_copies":8078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12924,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15788","reported_seq":15932,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.080746+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:24.080746+0000","last_peered":"2024-09-16T04:56:24.080746+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:24.080746+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16666648,"num_objects":4070,"num_object_clones":0,"num_object_copies":8140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4070,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12527,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":751}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'16120","reported_seq":16196,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:24.032857+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:24.032857+0000","last_peered":"2024-09-16T04:56:24.032857+0000","last_clean":"2024-09-16T04:56:24.032857+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:24.032857+0000","last_undegraded":"2024-09-16T04:56:24.032857+0000","last_fullsized":"2024-09-16T04:56:24.032857+0000","mapping_epoch":35,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17301504,"num_objects":4224,"num_object_clones":0,"num_object_copies":8448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16120,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'15874","reported_seq":16099,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:19.448611+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:19.448611+0000","last_peered":"2024-09-16T04:56:19.448611+0000","last_clean":"2024-09-16T04:56:19.448611+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:19.448611+0000","last_undegraded":"2024-09-16T04:56:19.448611+0000","last_fullsized":"2024-09-16T04:56:19.448611+0000","mapping_epoch":34,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17489920,"num_objects":4270,"num_object_clones":0,"num_object_copies":8540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12906,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16022","reported_seq":16161,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:19.440469+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:19.440469+0000","last_peered":"2024-09-16T04:56:19.440469+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:19.440469+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17575936,"num_objects":4291,"num_object_clones":0,"num_object_copies":8582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13015,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15661","reported_seq":15815,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:19.362009+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:19.362009+0000","last_peered":"2024-09-16T04:56:19.362009+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:19.362009+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5600","ondisk_log_start":"45'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17199104,"num_objects":4199,"num_object_clones":0,"num_object_copies":8398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12316,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":800}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15718","reported_seq":15940,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:19.380984+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:19.380984+0000","last_peered":"2024-09-16T04:56:19.380984+0000","last_clean":"2024-09-16T04:56:19.380984+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:19.380984+0000","last_undegraded":"2024-09-16T04:56:19.380984+0000","last_fullsized":"2024-09-16T04:56:19.380984+0000","mapping_epoch":34,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17604608,"num_objects":4298,"num_object_clones":0,"num_object_copies":8596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12788,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15910","reported_seq":16144,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.084972+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:24.084972+0000","last_peered":"2024-09-16T04:56:24.084972+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:24.084972+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17084416,"num_objects":4171,"num_object_clones":0,"num_object_copies":8342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12661,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":850}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16003","reported_seq":16134,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:19.381098+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:19.381098+0000","last_peered":"2024-09-16T04:56:19.381098+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:19.381098+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17264640,"num_objects":4215,"num_object_clones":0,"num_object_copies":8430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12658,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":750}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15851","reported_seq":21822,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.152055+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:24.152055+0000","last_peered":"2024-09-16T04:56:24.152055+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:24.152055+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16909522,"num_objects":4130,"num_object_clones":0,"num_object_copies":8260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":437,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15823,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2863,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3693}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15909","reported_seq":22771,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:24.071305+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:24.071305+0000","last_peered":"2024-09-16T04:56:24.071305+0000","last_clean":"2024-09-16T04:56:24.071305+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:24.071305+0000","last_undegraded":"2024-09-16T04:56:24.071305+0000","last_fullsized":"2024-09-16T04:56:24.071305+0000","mapping_epoch":46,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16666624,"num_objects":4069,"num_object_clones":0,"num_object_copies":8138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12632,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15827","reported_seq":15931,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.113936+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:24.113936+0000","last_peered":"2024-09-16T04:56:24.113936+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:24.113936+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16585321,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15826,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":643}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16294","reported_seq":16402,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.125022+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:24.125022+0000","last_peered":"2024-09-16T04:56:24.125022+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:24.125022+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17360699,"num_objects":4241,"num_object_clones":0,"num_object_copies":8482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16291,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":727}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16008","reported_seq":22808,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:23.631743+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:23.631743+0000","last_peered":"2024-09-16T04:56:23.631743+0000","last_clean":"2024-09-16T04:56:23.631743+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:23.631743+0000","last_undegraded":"2024-09-16T04:56:23.631743+0000","last_fullsized":"2024-09-16T04:56:23.631743+0000","mapping_epoch":48,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17162240,"num_objects":4190,"num_object_clones":0,"num_object_copies":8380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12762,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16135","reported_seq":16281,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:19.428467+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:19.428467+0000","last_peered":"2024-09-16T04:56:19.428467+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:19.428467+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17535593,"num_objects":4282,"num_object_clones":0,"num_object_copies":8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12904,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":273961207,"num_objects":66892,"num_object_clones":0,"num_object_copies":133784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66892,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219738,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9543,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712962048,"data_stored":712887803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160660,"ondisk_log_size":160660,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622688,"kb_used_data":248076,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93749152,"statfs":{"total":96636764160,"available":95999131648,"internally_reserved":0,"allocated":254029824,"data_stored":253746498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":383577769},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,6,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593810,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552032,"kb_used_data":199632,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93819808,"statfs":{"total":96636764160,"available":96071483392,"internally_reserved":0,"allocated":204423168,"data_stored":204084178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":360836782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634312,"kb_used_data":251444,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93737528,"statfs":{"total":96636764160,"available":95987228672,"internally_reserved":0,"allocated":257478656,"data_stored":257199402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257052672,"data_stored":257049193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":202895360,"data_stored":202832145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":253014016,"data_stored":253006465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:24.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:56:24.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'15585", 'reported_seq': 15670, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:19.413074+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:56:19.413074+0000', 'last_peered': '2024-09-16T04:56:19.413074+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:56:19.413074+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'5500", 'ondisk_log_start': "45'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17010688, 'num_objects': 4153, 'num_object_clones': 0, 'num_object_copies': 8306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4153, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15585, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 812}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:24.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:56:24.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'15788", 'reported_seq': 15932, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:24.080746+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:56:24.080746+0000', 'last_peered': '2024-09-16T04:56:24.080746+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:56:24.080746+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'5700", 'ondisk_log_start': "45'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16666648, 'num_objects': 4070, 'num_object_clones': 0, 'num_object_copies': 8140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4070, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12527, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 751}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:24.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:56:24.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'16022", 'reported_seq': 16161, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:19.440469+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:56:19.440469+0000', 'last_peered': '2024-09-16T04:56:19.440469+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:56:19.440469+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'6000", 'ondisk_log_start': "45'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17575936, 'num_objects': 4291, 'num_object_clones': 0, 'num_object_copies': 8582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4291, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13015, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 835}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:24.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:56:24.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'15661", 'reported_seq': 15815, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:19.362009+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:56:19.362009+0000', 'last_peered': '2024-09-16T04:56:19.362009+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:56:19.362009+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'5600", 'ondisk_log_start': "45'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17199104, 'num_objects': 4199, 'num_object_clones': 0, 'num_object_copies': 8398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12316, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 800}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:24.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:56:24.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'15910", 'reported_seq': 16144, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:24.084972+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:56:24.084972+0000', 'last_peered': '2024-09-16T04:56:24.084972+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:56:24.084972+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'5900", 'ondisk_log_start': "45'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17084416, 'num_objects': 4171, 'num_object_clones': 0, 'num_object_copies': 8342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4171, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12661, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 850}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:24.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:56:24.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'16003", 'reported_seq': 16134, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:19.381098+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:56:19.381098+0000', 'last_peered': '2024-09-16T04:56:19.381098+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:56:19.381098+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'6000", 'ondisk_log_start': "45'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17264640, 'num_objects': 4215, 'num_object_clones': 0, 'num_object_copies': 8430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4215, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12658, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 750}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:24.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:56:24.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'15851", 'reported_seq': 21822, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:24.152055+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:56:24.152055+0000', 'last_peered': '2024-09-16T04:56:24.152055+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:56:24.152055+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'5800", 'ondisk_log_start': "45'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16909522, 'num_objects': 4130, 'num_object_clones': 0, 'num_object_copies': 8260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 437, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15823, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2863, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3693}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:24.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:56:24.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'15827", 'reported_seq': 15931, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:24.113936+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:56:24.113936+0000', 'last_peered': '2024-09-16T04:56:24.113936+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:56:24.113936+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'5800", 'ondisk_log_start': "45'5800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16585321, 'num_objects': 4050, 'num_object_clones': 0, 'num_object_copies': 8100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4050, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15826, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 643}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:24.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:56:24.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'16294", 'reported_seq': 16402, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:24.125022+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:56:24.125022+0000', 'last_peered': '2024-09-16T04:56:24.125022+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:56:24.125022+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'6200", 'ondisk_log_start': "45'6200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17360699, 'num_objects': 4241, 'num_object_clones': 0, 'num_object_copies': 8482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4241, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16291, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 727}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:24.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:56:24.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'16135", 'reported_seq': 16281, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:19.428467+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:56:19.428467+0000', 'last_peered': '2024-09-16T04:56:19.428467+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:56:19.428467+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'6100", 'ondisk_log_start': "45'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17535593, 'num_objects': 4282, 'num_object_clones': 0, 'num_object_copies': 8564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4282, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12904, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 758}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:24.710 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:56:25.865 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:25.865 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31199/133788 objects degraded (23.320%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":533,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":66894,"data_bytes":273961226,"bytes_used":1852448768,"bytes_avail":288057843712,"bytes_total":289910292480,"degraded_objects":31199,"degraded_total":133788,"degraded_ratio":0.23319729721649179,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":348,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":501},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:56:25.918 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:56:25.918 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:26.860 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:26.905 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-16T04:56:26.456241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":269758730,"num_objects":65868,"num_object_clones":0,"num_object_copies":131736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65868,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220766,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9544,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160689,"ondisk_log_size":160689,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1794948,"kb_used_data":693392,"kb_used_omap":12,"kb_used_meta":1101491,"kb_avail":281320572,"statfs":{"total":289910292480,"available":288072265728,"internally_reserved":0,"allocated":710033408,"data_stored":709131838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1127926785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,3,7,14,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23425024,"num_objects":-5719,"num_object_clones":0,"num_object_copies":-11438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5719,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004087"},"pg_stats":[{"pgid":"2.f","version":"49'15732","reported_seq":15814,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.421410+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:24.421410+0000","last_peered":"2024-09-16T04:56:24.421410+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:24.421410+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16408576,"num_objects":4006,"num_object_clones":0,"num_object_copies":8012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15732,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":665}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'15955","reported_seq":16075,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:24.071917+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:24.071917+0000","last_peered":"2024-09-16T04:56:24.071917+0000","last_clean":"2024-09-16T04:56:24.071917+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:24.071917+0000","last_undegraded":"2024-09-16T04:56:24.071917+0000","last_fullsized":"2024-09-16T04:56:24.071917+0000","mapping_epoch":35,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16543744,"num_objects":4039,"num_object_clones":0,"num_object_copies":8078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12924,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15788","reported_seq":15932,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.080746+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:24.080746+0000","last_peered":"2024-09-16T04:56:24.080746+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:24.080746+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5700","ondisk_log_start":"45'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16666648,"num_objects":4070,"num_object_clones":0,"num_object_copies":8140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4070,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12527,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":751}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'16120","reported_seq":16196,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:24.032857+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:24.032857+0000","last_peered":"2024-09-16T04:56:24.032857+0000","last_clean":"2024-09-16T04:56:24.032857+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:24.032857+0000","last_undegraded":"2024-09-16T04:56:24.032857+0000","last_fullsized":"2024-09-16T04:56:24.032857+0000","mapping_epoch":35,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17301504,"num_objects":4224,"num_object_clones":0,"num_object_copies":8448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16120,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16012","reported_seq":16237,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:24.458340+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:24.458340+0000","last_peered":"2024-09-16T04:56:24.458340+0000","last_clean":"2024-09-16T04:56:24.458340+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:24.458340+0000","last_undegraded":"2024-09-16T04:56:24.458340+0000","last_fullsized":"2024-09-16T04:56:24.458340+0000","mapping_epoch":34,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16924672,"num_objects":4132,"num_object_clones":0,"num_object_copies":8264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13044,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16167","reported_seq":16306,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.399759+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:24.399759+0000","last_peered":"2024-09-16T04:56:24.399759+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:24.399759+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16982016,"num_objects":4146,"num_object_clones":0,"num_object_copies":8292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13160,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":690}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15805","reported_seq":15959,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.424967+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:24.424967+0000","last_peered":"2024-09-16T04:56:24.424967+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:24.424967+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16609280,"num_objects":4055,"num_object_clones":0,"num_object_copies":8110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12460,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":656}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'15879","reported_seq":16101,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:24.426099+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:24.426099+0000","last_peered":"2024-09-16T04:56:24.426099+0000","last_clean":"2024-09-16T04:56:24.426099+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:24.426099+0000","last_undegraded":"2024-09-16T04:56:24.426099+0000","last_fullsized":"2024-09-16T04:56:24.426099+0000","mapping_epoch":34,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16945152,"num_objects":4137,"num_object_clones":0,"num_object_copies":8274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12949,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'15910","reported_seq":16144,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.084972+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:24.084972+0000","last_peered":"2024-09-16T04:56:24.084972+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:24.084972+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17084416,"num_objects":4171,"num_object_clones":0,"num_object_copies":8342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12661,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":850}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16155","reported_seq":16291,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.345158+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:24.345158+0000","last_peered":"2024-09-16T04:56:24.345158+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:24.345158+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16642048,"num_objects":4063,"num_object_clones":0,"num_object_copies":8126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12810,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15851","reported_seq":21822,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.152055+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:24.152055+0000","last_peered":"2024-09-16T04:56:24.152055+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:24.152055+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16909522,"num_objects":4130,"num_object_clones":0,"num_object_copies":8260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":437,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15823,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2863,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3693}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'15909","reported_seq":22771,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:24.071305+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:24.071305+0000","last_peered":"2024-09-16T04:56:24.071305+0000","last_clean":"2024-09-16T04:56:24.071305+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:24.071305+0000","last_undegraded":"2024-09-16T04:56:24.071305+0000","last_fullsized":"2024-09-16T04:56:24.071305+0000","mapping_epoch":46,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16666624,"num_objects":4069,"num_object_clones":0,"num_object_copies":8138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12632,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15827","reported_seq":15931,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.113936+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:24.113936+0000","last_peered":"2024-09-16T04:56:24.113936+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:24.113936+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16585321,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15826,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":643}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16294","reported_seq":16402,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.125022+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:24.125022+0000","last_peered":"2024-09-16T04:56:24.125022+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:24.125022+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17360699,"num_objects":4241,"num_object_clones":0,"num_object_copies":8482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16291,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":727}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16008","reported_seq":22808,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:23.631743+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:23.631743+0000","last_peered":"2024-09-16T04:56:23.631743+0000","last_clean":"2024-09-16T04:56:23.631743+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:23.631743+0000","last_undegraded":"2024-09-16T04:56:23.631743+0000","last_fullsized":"2024-09-16T04:56:23.631743+0000","mapping_epoch":48,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17162240,"num_objects":4190,"num_object_clones":0,"num_object_copies":8380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12762,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16274","reported_seq":16420,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:24.467175+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:24.467175+0000","last_peered":"2024-09-16T04:56:24.467175+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:24.467175+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16966249,"num_objects":4143,"num_object_clones":0,"num_object_copies":8286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13043,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":619}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":269758711,"num_objects":65866,"num_object_clones":0,"num_object_copies":131732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65866,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220764,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9543,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":706936832,"data_stored":706862587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622688,"kb_used_data":248076,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93749152,"statfs":{"total":96636764160,"available":95999131648,"internally_reserved":0,"allocated":254029824,"data_stored":253746498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":383577769},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,6,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593810,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552032,"kb_used_data":199632,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93819808,"statfs":{"total":96636764160,"available":96071483392,"internally_reserved":0,"allocated":204423168,"data_stored":204084178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":360836782},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620228,"kb_used_data":245684,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93751612,"statfs":{"total":96636764160,"available":96001650688,"internally_reserved":0,"allocated":251580416,"data_stored":251301162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":251027456,"data_stored":251023977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":202895360,"data_stored":202832145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":253014016,"data_stored":253006465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:29.907 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:30.658 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:30.659 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:30.698 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-16T04:56:30.464805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":259969290,"num_objects":63478,"num_object_clones":0,"num_object_copies":126956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63478,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223156,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9688,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160879,"ondisk_log_size":160879,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1788136,"kb_used_data":674104,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281327384,"statfs":{"total":289910292480,"available":288079241216,"internally_reserved":0,"allocated":690282496,"data_stored":689380926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1140706305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,4,3,12,7,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29171712,"num_objects":-7122,"num_object_clones":0,"num_object_copies":-14244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-374,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010354"},"pg_stats":[{"pgid":"2.f","version":"49'15885","reported_seq":15970,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.484456+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:29.484456+0000","last_peered":"2024-09-16T04:56:29.484456+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:29.484456+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15781888,"num_objects":3853,"num_object_clones":0,"num_object_copies":7706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15885,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":512}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16104","reported_seq":16226,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.155951+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:29.155951+0000","last_peered":"2024-09-16T04:56:29.155951+0000","last_clean":"2024-09-16T04:56:29.155951+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:29.155951+0000","last_undegraded":"2024-09-16T04:56:29.155951+0000","last_fullsized":"2024-09-16T04:56:29.155951+0000","mapping_epoch":35,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15933440,"num_objects":3890,"num_object_clones":0,"num_object_copies":7780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13073,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15950","reported_seq":16093,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.094055+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:29.094055+0000","last_peered":"2024-09-16T04:56:29.094055+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:29.094055+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16003096,"num_objects":3908,"num_object_clones":0,"num_object_copies":7816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3908,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12689,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":589}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'16270","reported_seq":16346,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.066119+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:29.066119+0000","last_peered":"2024-09-16T04:56:29.066119+0000","last_clean":"2024-09-16T04:56:29.066119+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:29.066119+0000","last_undegraded":"2024-09-16T04:56:29.066119+0000","last_fullsized":"2024-09-16T04:56:29.066119+0000","mapping_epoch":35,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16687104,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16270,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16190","reported_seq":16415,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.392883+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:29.392883+0000","last_peered":"2024-09-16T04:56:29.392883+0000","last_clean":"2024-09-16T04:56:29.392883+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:29.392883+0000","last_undegraded":"2024-09-16T04:56:29.392883+0000","last_fullsized":"2024-09-16T04:56:29.392883+0000","mapping_epoch":34,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16195584,"num_objects":3954,"num_object_clones":0,"num_object_copies":7908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13222,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16301","reported_seq":16437,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.474112+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:29.474112+0000","last_peered":"2024-09-16T04:56:29.474112+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:29.474112+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16433152,"num_objects":4012,"num_object_clones":0,"num_object_copies":8024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13294,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15937","reported_seq":16087,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.402564+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:29.402564+0000","last_peered":"2024-09-16T04:56:29.402564+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:29.402564+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16068608,"num_objects":3923,"num_object_clones":0,"num_object_copies":7846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12592,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16035","reported_seq":16257,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.408875+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:29.408875+0000","last_peered":"2024-09-16T04:56:29.408875+0000","last_clean":"2024-09-16T04:56:29.408875+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:29.408875+0000","last_undegraded":"2024-09-16T04:56:29.408875+0000","last_fullsized":"2024-09-16T04:56:29.408875+0000","mapping_epoch":34,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16306176,"num_objects":3981,"num_object_clones":0,"num_object_copies":7962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13105,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16047","reported_seq":16281,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.141382+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:29.141382+0000","last_peered":"2024-09-16T04:56:29.141382+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:29.141382+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16523264,"num_objects":4034,"num_object_clones":0,"num_object_copies":8068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":713}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16297","reported_seq":16433,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.474433+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:29.474433+0000","last_peered":"2024-09-16T04:56:29.474433+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:29.474433+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12952,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15994","reported_seq":22253,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.187842+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:29.187842+0000","last_peered":"2024-09-16T04:56:29.187842+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:29.187842+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16323794,"num_objects":3987,"num_object_clones":0,"num_object_copies":7974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15966,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3007,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3694}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16063","reported_seq":22925,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.091705+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:29.091705+0000","last_peered":"2024-09-16T04:56:29.091705+0000","last_clean":"2024-09-16T04:56:29.091705+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:29.091705+0000","last_undegraded":"2024-09-16T04:56:29.091705+0000","last_fullsized":"2024-09-16T04:56:29.091705+0000","mapping_epoch":46,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16035840,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12786,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15965","reported_seq":16067,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.177638+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:29.177638+0000","last_peered":"2024-09-16T04:56:29.177638+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:29.177638+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16020073,"num_objects":3912,"num_object_clones":0,"num_object_copies":7824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15964,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16454","reported_seq":16563,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.123919+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:29.123919+0000","last_peered":"2024-09-16T04:56:29.123919+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:29.123919+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16705339,"num_objects":4081,"num_object_clones":0,"num_object_copies":8162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16451,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16170","reported_seq":22969,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:28.648746+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:28.648746+0000","last_peered":"2024-09-16T04:56:28.648746+0000","last_clean":"2024-09-16T04:56:28.648746+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:28.648746+0000","last_undegraded":"2024-09-16T04:56:28.648746+0000","last_fullsized":"2024-09-16T04:56:28.648746+0000","mapping_epoch":48,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16498688,"num_objects":4028,"num_object_clones":0,"num_object_copies":8056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12924,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16414","reported_seq":16559,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.439731+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:29.439731+0000","last_peered":"2024-09-16T04:56:29.439731+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:29.439731+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16392809,"num_objects":4003,"num_object_clones":0,"num_object_copies":8006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13183,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":259969271,"num_objects":63476,"num_object_clones":0,"num_object_copies":126952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223154,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9687,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":687534080,"data_stored":687459835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160876,"ondisk_log_size":160876,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620560,"kb_used_data":241792,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93751280,"statfs":{"total":96636764160,"available":96001310720,"internally_reserved":0,"allocated":247595008,"data_stored":247311682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":387837609},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,2,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593811,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549224,"kb_used_data":192664,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93822616,"statfs":{"total":96636764160,"available":96074358784,"internally_reserved":0,"allocated":197287936,"data_stored":196948946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365096622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618352,"kb_used_data":239648,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93753488,"statfs":{"total":96636764160,"available":96003571712,"internally_reserved":0,"allocated":245399552,"data_stored":245120298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":244944896,"data_stored":244941417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":195813376,"data_stored":195750161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":246775808,"data_stored":246768257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:30.700 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:31.423 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:31.499 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-16T04:56:30.464805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":259969290,"num_objects":63478,"num_object_clones":0,"num_object_copies":126956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63478,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223156,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9688,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160879,"ondisk_log_size":160879,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1788136,"kb_used_data":674104,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281327384,"statfs":{"total":289910292480,"available":288079241216,"internally_reserved":0,"allocated":690282496,"data_stored":689380926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1140706305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,4,3,12,7,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29171712,"num_objects":-7122,"num_object_clones":0,"num_object_copies":-14244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-374,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010354"},"pg_stats":[{"pgid":"2.f","version":"49'15885","reported_seq":15970,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.484456+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:29.484456+0000","last_peered":"2024-09-16T04:56:29.484456+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:29.484456+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15781888,"num_objects":3853,"num_object_clones":0,"num_object_copies":7706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15885,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":512}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16104","reported_seq":16226,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.155951+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:29.155951+0000","last_peered":"2024-09-16T04:56:29.155951+0000","last_clean":"2024-09-16T04:56:29.155951+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:29.155951+0000","last_undegraded":"2024-09-16T04:56:29.155951+0000","last_fullsized":"2024-09-16T04:56:29.155951+0000","mapping_epoch":35,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15933440,"num_objects":3890,"num_object_clones":0,"num_object_copies":7780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13073,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15950","reported_seq":16093,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.094055+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:29.094055+0000","last_peered":"2024-09-16T04:56:29.094055+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:29.094055+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16003096,"num_objects":3908,"num_object_clones":0,"num_object_copies":7816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3908,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12689,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":589}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'16270","reported_seq":16346,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.066119+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:29.066119+0000","last_peered":"2024-09-16T04:56:29.066119+0000","last_clean":"2024-09-16T04:56:29.066119+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:29.066119+0000","last_undegraded":"2024-09-16T04:56:29.066119+0000","last_fullsized":"2024-09-16T04:56:29.066119+0000","mapping_epoch":35,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16687104,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16270,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16190","reported_seq":16415,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.392883+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:29.392883+0000","last_peered":"2024-09-16T04:56:29.392883+0000","last_clean":"2024-09-16T04:56:29.392883+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:29.392883+0000","last_undegraded":"2024-09-16T04:56:29.392883+0000","last_fullsized":"2024-09-16T04:56:29.392883+0000","mapping_epoch":34,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16195584,"num_objects":3954,"num_object_clones":0,"num_object_copies":7908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13222,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16301","reported_seq":16437,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.474112+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:29.474112+0000","last_peered":"2024-09-16T04:56:29.474112+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:29.474112+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16433152,"num_objects":4012,"num_object_clones":0,"num_object_copies":8024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13294,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15937","reported_seq":16087,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.402564+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:29.402564+0000","last_peered":"2024-09-16T04:56:29.402564+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:29.402564+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16068608,"num_objects":3923,"num_object_clones":0,"num_object_copies":7846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12592,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16035","reported_seq":16257,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.408875+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:29.408875+0000","last_peered":"2024-09-16T04:56:29.408875+0000","last_clean":"2024-09-16T04:56:29.408875+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:29.408875+0000","last_undegraded":"2024-09-16T04:56:29.408875+0000","last_fullsized":"2024-09-16T04:56:29.408875+0000","mapping_epoch":34,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16306176,"num_objects":3981,"num_object_clones":0,"num_object_copies":7962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13105,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16047","reported_seq":16281,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.141382+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:29.141382+0000","last_peered":"2024-09-16T04:56:29.141382+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:29.141382+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16523264,"num_objects":4034,"num_object_clones":0,"num_object_copies":8068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":713}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16297","reported_seq":16433,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.474433+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:29.474433+0000","last_peered":"2024-09-16T04:56:29.474433+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:29.474433+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12952,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15994","reported_seq":22253,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.187842+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:29.187842+0000","last_peered":"2024-09-16T04:56:29.187842+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:29.187842+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16323794,"num_objects":3987,"num_object_clones":0,"num_object_copies":7974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15966,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3007,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3694}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16063","reported_seq":22925,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.091705+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:29.091705+0000","last_peered":"2024-09-16T04:56:29.091705+0000","last_clean":"2024-09-16T04:56:29.091705+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:29.091705+0000","last_undegraded":"2024-09-16T04:56:29.091705+0000","last_fullsized":"2024-09-16T04:56:29.091705+0000","mapping_epoch":46,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16035840,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12786,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15965","reported_seq":16067,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.177638+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:29.177638+0000","last_peered":"2024-09-16T04:56:29.177638+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:29.177638+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16020073,"num_objects":3912,"num_object_clones":0,"num_object_copies":7824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15964,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16454","reported_seq":16563,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.123919+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:29.123919+0000","last_peered":"2024-09-16T04:56:29.123919+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:29.123919+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16705339,"num_objects":4081,"num_object_clones":0,"num_object_copies":8162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16451,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16170","reported_seq":22969,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:28.648746+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:28.648746+0000","last_peered":"2024-09-16T04:56:28.648746+0000","last_clean":"2024-09-16T04:56:28.648746+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:28.648746+0000","last_undegraded":"2024-09-16T04:56:28.648746+0000","last_fullsized":"2024-09-16T04:56:28.648746+0000","mapping_epoch":48,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16498688,"num_objects":4028,"num_object_clones":0,"num_object_copies":8056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12924,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16414","reported_seq":16559,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.439731+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:29.439731+0000","last_peered":"2024-09-16T04:56:29.439731+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:29.439731+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16392809,"num_objects":4003,"num_object_clones":0,"num_object_copies":8006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13183,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":259969271,"num_objects":63476,"num_object_clones":0,"num_object_copies":126952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223154,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9687,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":687534080,"data_stored":687459835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160876,"ondisk_log_size":160876,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620560,"kb_used_data":241792,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93751280,"statfs":{"total":96636764160,"available":96001310720,"internally_reserved":0,"allocated":247595008,"data_stored":247311682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":387837609},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,2,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593811,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549224,"kb_used_data":192664,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93822616,"statfs":{"total":96636764160,"available":96074358784,"internally_reserved":0,"allocated":197287936,"data_stored":196948946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365096622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618352,"kb_used_data":239648,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93753488,"statfs":{"total":96636764160,"available":96003571712,"internally_reserved":0,"allocated":245399552,"data_stored":245120298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":244944896,"data_stored":244941417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":195813376,"data_stored":195750161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":246775808,"data_stored":246768257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:31.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:56:31.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'15885", 'reported_seq': 15970, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:29.484456+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:56:29.484456+0000', 'last_peered': '2024-09-16T04:56:29.484456+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:56:29.484456+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'5800", 'ondisk_log_start': "45'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15781888, 'num_objects': 3853, 'num_object_clones': 0, 'num_object_copies': 7706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15885, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 512}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:31.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:56:31.500 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'15950", 'reported_seq': 16093, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:29.094055+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:56:29.094055+0000', 'last_peered': '2024-09-16T04:56:29.094055+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:56:29.094055+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'5900", 'ondisk_log_start': "45'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16003096, 'num_objects': 3908, 'num_object_clones': 0, 'num_object_copies': 7816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3908, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12689, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 589}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:56:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'16301", 'reported_seq': 16437, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:29.474112+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:56:29.474112+0000', 'last_peered': '2024-09-16T04:56:29.474112+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:56:29.474112+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'6300", 'ondisk_log_start': "45'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16433152, 'num_objects': 4012, 'num_object_clones': 0, 'num_object_copies': 8024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4012, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13294, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 556}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:56:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'15937", 'reported_seq': 16087, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:29.402564+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:56:29.402564+0000', 'last_peered': '2024-09-16T04:56:29.402564+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:56:29.402564+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'5900", 'ondisk_log_start': "45'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16068608, 'num_objects': 3923, 'num_object_clones': 0, 'num_object_copies': 7846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3923, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12592, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:56:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'16047", 'reported_seq': 16281, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:29.141382+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:56:29.141382+0000', 'last_peered': '2024-09-16T04:56:29.141382+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:56:29.141382+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'6000", 'ondisk_log_start': "45'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16523264, 'num_objects': 4034, 'num_object_clones': 0, 'num_object_copies': 8068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4034, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12798, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:56:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'16297", 'reported_seq': 16433, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:29.474433+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:56:29.474433+0000', 'last_peered': '2024-09-16T04:56:29.474433+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:56:29.474433+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'6200", 'ondisk_log_start': "45'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16060416, 'num_objects': 3921, 'num_object_clones': 0, 'num_object_copies': 7842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3921, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12952, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 456}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:56:31.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'15994", 'reported_seq': 22253, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:29.187842+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:56:29.187842+0000', 'last_peered': '2024-09-16T04:56:29.187842+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:56:29.187842+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'5900", 'ondisk_log_start': "45'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16323794, 'num_objects': 3987, 'num_object_clones': 0, 'num_object_copies': 7974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 293, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3987, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15966, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3007, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3694}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:31.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:56:31.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'15965", 'reported_seq': 16067, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:29.177638+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:56:29.177638+0000', 'last_peered': '2024-09-16T04:56:29.177638+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:56:29.177638+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'5900", 'ondisk_log_start': "45'5900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16020073, 'num_objects': 3912, 'num_object_clones': 0, 'num_object_copies': 7824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3912, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15964, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 505}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:31.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:56:31.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'16454", 'reported_seq': 16563, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:29.123919+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:56:29.123919+0000', 'last_peered': '2024-09-16T04:56:29.123919+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:56:29.123919+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'6400", 'ondisk_log_start': "45'6400", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16705339, 'num_objects': 4081, 'num_object_clones': 0, 'num_object_copies': 8162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4081, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16451, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 567}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:31.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:56:31.502 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'16414", 'reported_seq': 16559, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:29.439731+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:56:29.439731+0000', 'last_peered': '2024-09-16T04:56:29.439731+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:56:29.439731+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'6400", 'ondisk_log_start': "45'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16392809, 'num_objects': 4003, 'num_object_clones': 0, 'num_object_copies': 8006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4003, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13183, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 479}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:31.502 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:56:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:32.456 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31055/126956 objects degraded (24.461%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":540,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":63478,"data_bytes":259969290,"bytes_used":1831051264,"bytes_avail":288079241216,"bytes_total":289910292480,"degraded_objects":31055,"degraded_total":126956,"degraded_ratio":0.24461230662591768,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":374,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":592},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:56:32.510 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:56:32.510 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:33.386 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:33.389 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:33.443 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-16T04:56:32.465290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":259969290,"num_objects":63478,"num_object_clones":0,"num_object_copies":126956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63478,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223156,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9688,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160879,"ondisk_log_size":160879,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1788136,"kb_used_data":674104,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281327384,"statfs":{"total":289910292480,"available":288079241216,"internally_reserved":0,"allocated":690282496,"data_stored":689380926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1140706305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,4,3,12,7,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19525632,"num_objects":-4767,"num_object_clones":0,"num_object_copies":-9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4767,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010412"},"pg_stats":[{"pgid":"2.f","version":"49'15885","reported_seq":15970,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.484456+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:29.484456+0000","last_peered":"2024-09-16T04:56:29.484456+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:29.484456+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'5800","ondisk_log_start":"45'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15781888,"num_objects":3853,"num_object_clones":0,"num_object_copies":7706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15885,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":512}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16104","reported_seq":16226,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.155951+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:29.155951+0000","last_peered":"2024-09-16T04:56:29.155951+0000","last_clean":"2024-09-16T04:56:29.155951+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:29.155951+0000","last_undegraded":"2024-09-16T04:56:29.155951+0000","last_fullsized":"2024-09-16T04:56:29.155951+0000","mapping_epoch":35,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15933440,"num_objects":3890,"num_object_clones":0,"num_object_copies":7780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13073,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'15950","reported_seq":16093,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.094055+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:29.094055+0000","last_peered":"2024-09-16T04:56:29.094055+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:29.094055+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16003096,"num_objects":3908,"num_object_clones":0,"num_object_copies":7816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3908,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12689,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":589}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'16270","reported_seq":16346,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.066119+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:29.066119+0000","last_peered":"2024-09-16T04:56:29.066119+0000","last_clean":"2024-09-16T04:56:29.066119+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:29.066119+0000","last_undegraded":"2024-09-16T04:56:29.066119+0000","last_fullsized":"2024-09-16T04:56:29.066119+0000","mapping_epoch":35,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16687104,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16270,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16190","reported_seq":16415,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.392883+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:29.392883+0000","last_peered":"2024-09-16T04:56:29.392883+0000","last_clean":"2024-09-16T04:56:29.392883+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:29.392883+0000","last_undegraded":"2024-09-16T04:56:29.392883+0000","last_fullsized":"2024-09-16T04:56:29.392883+0000","mapping_epoch":34,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16195584,"num_objects":3954,"num_object_clones":0,"num_object_copies":7908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13222,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16301","reported_seq":16437,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.474112+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:29.474112+0000","last_peered":"2024-09-16T04:56:29.474112+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:29.474112+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16433152,"num_objects":4012,"num_object_clones":0,"num_object_copies":8024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13294,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'15937","reported_seq":16087,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.402564+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:29.402564+0000","last_peered":"2024-09-16T04:56:29.402564+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:29.402564+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16068608,"num_objects":3923,"num_object_clones":0,"num_object_copies":7846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12592,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":524}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16035","reported_seq":16257,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.408875+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:29.408875+0000","last_peered":"2024-09-16T04:56:29.408875+0000","last_clean":"2024-09-16T04:56:29.408875+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:29.408875+0000","last_undegraded":"2024-09-16T04:56:29.408875+0000","last_fullsized":"2024-09-16T04:56:29.408875+0000","mapping_epoch":34,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16306176,"num_objects":3981,"num_object_clones":0,"num_object_copies":7962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13105,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16047","reported_seq":16281,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.141382+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:29.141382+0000","last_peered":"2024-09-16T04:56:29.141382+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:29.141382+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16523264,"num_objects":4034,"num_object_clones":0,"num_object_copies":8068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":713}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16297","reported_seq":16433,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.474433+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:29.474433+0000","last_peered":"2024-09-16T04:56:29.474433+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:29.474433+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16060416,"num_objects":3921,"num_object_clones":0,"num_object_copies":7842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12952,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'15994","reported_seq":22253,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.187842+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:29.187842+0000","last_peered":"2024-09-16T04:56:29.187842+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:29.187842+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16323794,"num_objects":3987,"num_object_clones":0,"num_object_copies":7974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15966,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3007,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3694}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16063","reported_seq":22925,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:29.091705+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:29.091705+0000","last_peered":"2024-09-16T04:56:29.091705+0000","last_clean":"2024-09-16T04:56:29.091705+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:29.091705+0000","last_undegraded":"2024-09-16T04:56:29.091705+0000","last_fullsized":"2024-09-16T04:56:29.091705+0000","mapping_epoch":46,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16035840,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12786,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'15965","reported_seq":16067,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.177638+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:29.177638+0000","last_peered":"2024-09-16T04:56:29.177638+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:29.177638+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'5900","ondisk_log_start":"45'5900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16020073,"num_objects":3912,"num_object_clones":0,"num_object_copies":7824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15964,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16454","reported_seq":16563,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.123919+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:29.123919+0000","last_peered":"2024-09-16T04:56:29.123919+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:29.123919+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16705339,"num_objects":4081,"num_object_clones":0,"num_object_copies":8162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16451,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16170","reported_seq":22969,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:28.648746+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:28.648746+0000","last_peered":"2024-09-16T04:56:28.648746+0000","last_clean":"2024-09-16T04:56:28.648746+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:28.648746+0000","last_undegraded":"2024-09-16T04:56:28.648746+0000","last_fullsized":"2024-09-16T04:56:28.648746+0000","mapping_epoch":48,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16498688,"num_objects":4028,"num_object_clones":0,"num_object_copies":8056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12924,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16414","reported_seq":16559,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:29.439731+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:29.439731+0000","last_peered":"2024-09-16T04:56:29.439731+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:29.439731+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16392809,"num_objects":4003,"num_object_clones":0,"num_object_copies":8006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13183,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":259969271,"num_objects":63476,"num_object_clones":0,"num_object_copies":126952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223154,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9687,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":687534080,"data_stored":687459835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160876,"ondisk_log_size":160876,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620560,"kb_used_data":241792,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93751280,"statfs":{"total":96636764160,"available":96001310720,"internally_reserved":0,"allocated":247595008,"data_stored":247311682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":387837609},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,2,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593811,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549224,"kb_used_data":192664,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93822616,"statfs":{"total":96636764160,"available":96074358784,"internally_reserved":0,"allocated":197287936,"data_stored":196948946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":365096622},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618352,"kb_used_data":239648,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93753488,"statfs":{"total":96636764160,"available":96003571712,"internally_reserved":0,"allocated":245399552,"data_stored":245120298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":244944896,"data_stored":244941417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":195813376,"data_stored":195750161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":246775808,"data_stored":246768257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:36.448 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:37.185 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:37.275 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-16T04:56:36.466200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":250683658,"num_objects":61211,"num_object_clones":0,"num_object_copies":122422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30936,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61211,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225423,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9807,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160946,"ondisk_log_size":160946,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1774884,"kb_used_data":656688,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281340636,"statfs":{"total":289910292480,"available":288092811264,"internally_reserved":0,"allocated":672448512,"data_stored":671546942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1144966145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,5,2,3,8,6,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23277568,"num_objects":-5683,"num_object_clones":0,"num_object_copies":-11366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5683,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010428"},"pg_stats":[{"pgid":"2.f","version":"49'16011","reported_seq":16096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.451963+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:34.451963+0000","last_peered":"2024-09-16T04:56:34.451963+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:34.451963+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15265792,"num_objects":3727,"num_object_clones":0,"num_object_copies":7454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16011,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":386}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16246","reported_seq":16368,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.183340+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:34.183340+0000","last_peered":"2024-09-16T04:56:34.183340+0000","last_clean":"2024-09-16T04:56:34.183340+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:34.183340+0000","last_undegraded":"2024-09-16T04:56:34.183340+0000","last_fullsized":"2024-09-16T04:56:34.183340+0000","mapping_epoch":35,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15351808,"num_objects":3748,"num_object_clones":0,"num_object_copies":7496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13215,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16100","reported_seq":16245,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.192921+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:34.192921+0000","last_peered":"2024-09-16T04:56:34.192921+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:34.192921+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15388696,"num_objects":3758,"num_object_clones":0,"num_object_copies":7516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12839,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":439}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'16410","reported_seq":16486,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.179860+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:34.179860+0000","last_peered":"2024-09-16T04:56:34.179860+0000","last_clean":"2024-09-16T04:56:34.179860+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:34.179860+0000","last_undegraded":"2024-09-16T04:56:34.179860+0000","last_fullsized":"2024-09-16T04:56:34.179860+0000","mapping_epoch":35,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16113664,"num_objects":3934,"num_object_clones":0,"num_object_copies":7868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16410,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16346","reported_seq":16569,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.475159+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:34.475159+0000","last_peered":"2024-09-16T04:56:34.475159+0000","last_clean":"2024-09-16T04:56:34.475159+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:34.475159+0000","last_undegraded":"2024-09-16T04:56:34.475159+0000","last_fullsized":"2024-09-16T04:56:34.475159+0000","mapping_epoch":34,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15556608,"num_objects":3798,"num_object_clones":0,"num_object_copies":7596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13378,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16460","reported_seq":16597,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.502043+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:34.502043+0000","last_peered":"2024-09-16T04:56:34.502043+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:34.502043+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15781888,"num_objects":3853,"num_object_clones":0,"num_object_copies":7706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13453,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":397}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16092","reported_seq":16246,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.461174+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:34.461174+0000","last_peered":"2024-09-16T04:56:34.461174+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:34.461174+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15433728,"num_objects":3768,"num_object_clones":0,"num_object_copies":7536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12747,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16164","reported_seq":16386,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.393361+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:34.393361+0000","last_peered":"2024-09-16T04:56:34.393361+0000","last_clean":"2024-09-16T04:56:34.393361+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:34.393361+0000","last_undegraded":"2024-09-16T04:56:34.393361+0000","last_fullsized":"2024-09-16T04:56:34.393361+0000","mapping_epoch":34,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15777792,"num_objects":3852,"num_object_clones":0,"num_object_copies":7704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13234,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16196","reported_seq":16430,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.216046+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:34.216046+0000","last_peered":"2024-09-16T04:56:34.216046+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:34.216046+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15912960,"num_objects":3885,"num_object_clones":0,"num_object_copies":7770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12947,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":564}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16441","reported_seq":16577,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.469472+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:34.469472+0000","last_peered":"2024-09-16T04:56:34.469472+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:34.469472+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15470592,"num_objects":3777,"num_object_clones":0,"num_object_copies":7554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13096,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16138","reported_seq":22634,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.195250+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:34.195250+0000","last_peered":"2024-09-16T04:56:34.195250+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:34.195250+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15733970,"num_objects":3843,"num_object_clones":0,"num_object_copies":7686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16110,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3126,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3669}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16195","reported_seq":23057,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.086978+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:34.086978+0000","last_peered":"2024-09-16T04:56:34.086978+0000","last_clean":"2024-09-16T04:56:34.086978+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:34.086978+0000","last_undegraded":"2024-09-16T04:56:34.086978+0000","last_fullsized":"2024-09-16T04:56:34.086978+0000","mapping_epoch":46,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15495168,"num_objects":3783,"num_object_clones":0,"num_object_copies":7566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12918,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16095","reported_seq":16198,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.077896+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:34.077896+0000","last_peered":"2024-09-16T04:56:34.077896+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:34.077896+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15487593,"num_objects":3782,"num_object_clones":0,"num_object_copies":7564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16094,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":375}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16599","reported_seq":16708,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.176350+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:34.176350+0000","last_peered":"2024-09-16T04:56:34.176350+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:34.176350+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16111419,"num_objects":3936,"num_object_clones":0,"num_object_copies":7872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":422}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16309","reported_seq":23109,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:33.727700+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:33.727700+0000","last_peered":"2024-09-16T04:56:33.727700+0000","last_clean":"2024-09-16T04:56:33.727700+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:33.727700+0000","last_undegraded":"2024-09-16T04:56:33.727700+0000","last_fullsized":"2024-09-16T04:56:33.727700+0000","mapping_epoch":48,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15929344,"num_objects":3889,"num_object_clones":0,"num_object_copies":7778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13063,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16541","reported_seq":16687,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.462218+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:34.462218+0000","last_peered":"2024-09-16T04:56:34.462218+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:34.462218+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15872617,"num_objects":3876,"num_object_clones":0,"num_object_copies":7752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13310,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":352}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":250683639,"num_objects":61209,"num_object_clones":0,"num_object_copies":122418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30936,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61209,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225421,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9806,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":668778496,"data_stored":668704251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160943,"ondisk_log_size":160943,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606500,"kb_used_data":236052,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93765340,"statfs":{"total":96636764160,"available":96015708160,"internally_reserved":0,"allocated":241717248,"data_stored":241433922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":379317929},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593812,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550548,"kb_used_data":185668,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93821292,"statfs":{"total":96636764160,"available":96073003008,"internally_reserved":0,"allocated":190124032,"data_stored":189785042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":373616302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617836,"kb_used_data":234968,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93754004,"statfs":{"total":96636764160,"available":96004100096,"internally_reserved":0,"allocated":240607232,"data_stored":240327978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,2,4,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":239108096,"data_stored":239104617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188715008,"data_stored":188651793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":240955392,"data_stored":240947841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:37.277 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:38.237 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:38.238 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:38.271 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-16T04:56:36.466200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":250683658,"num_objects":61211,"num_object_clones":0,"num_object_copies":122422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30936,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61211,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225423,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9807,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160946,"ondisk_log_size":160946,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1774884,"kb_used_data":656688,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281340636,"statfs":{"total":289910292480,"available":288092811264,"internally_reserved":0,"allocated":672448512,"data_stored":671546942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1144966145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,5,2,3,8,6,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23277568,"num_objects":-5683,"num_object_clones":0,"num_object_copies":-11366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5683,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010428"},"pg_stats":[{"pgid":"2.f","version":"49'16011","reported_seq":16096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.451963+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:34.451963+0000","last_peered":"2024-09-16T04:56:34.451963+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:34.451963+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15265792,"num_objects":3727,"num_object_clones":0,"num_object_copies":7454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16011,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":386}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16246","reported_seq":16368,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.183340+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:34.183340+0000","last_peered":"2024-09-16T04:56:34.183340+0000","last_clean":"2024-09-16T04:56:34.183340+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:34.183340+0000","last_undegraded":"2024-09-16T04:56:34.183340+0000","last_fullsized":"2024-09-16T04:56:34.183340+0000","mapping_epoch":35,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15351808,"num_objects":3748,"num_object_clones":0,"num_object_copies":7496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13215,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16100","reported_seq":16245,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.192921+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:34.192921+0000","last_peered":"2024-09-16T04:56:34.192921+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:34.192921+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15388696,"num_objects":3758,"num_object_clones":0,"num_object_copies":7516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12839,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":439}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'16410","reported_seq":16486,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.179860+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:34.179860+0000","last_peered":"2024-09-16T04:56:34.179860+0000","last_clean":"2024-09-16T04:56:34.179860+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:34.179860+0000","last_undegraded":"2024-09-16T04:56:34.179860+0000","last_fullsized":"2024-09-16T04:56:34.179860+0000","mapping_epoch":35,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16113664,"num_objects":3934,"num_object_clones":0,"num_object_copies":7868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16410,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16346","reported_seq":16569,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.475159+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:34.475159+0000","last_peered":"2024-09-16T04:56:34.475159+0000","last_clean":"2024-09-16T04:56:34.475159+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:34.475159+0000","last_undegraded":"2024-09-16T04:56:34.475159+0000","last_fullsized":"2024-09-16T04:56:34.475159+0000","mapping_epoch":34,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15556608,"num_objects":3798,"num_object_clones":0,"num_object_copies":7596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13378,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16460","reported_seq":16597,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.502043+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:34.502043+0000","last_peered":"2024-09-16T04:56:34.502043+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:34.502043+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15781888,"num_objects":3853,"num_object_clones":0,"num_object_copies":7706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13453,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":397}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16092","reported_seq":16246,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.461174+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:34.461174+0000","last_peered":"2024-09-16T04:56:34.461174+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:34.461174+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15433728,"num_objects":3768,"num_object_clones":0,"num_object_copies":7536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12747,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16164","reported_seq":16386,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.393361+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:34.393361+0000","last_peered":"2024-09-16T04:56:34.393361+0000","last_clean":"2024-09-16T04:56:34.393361+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:34.393361+0000","last_undegraded":"2024-09-16T04:56:34.393361+0000","last_fullsized":"2024-09-16T04:56:34.393361+0000","mapping_epoch":34,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15777792,"num_objects":3852,"num_object_clones":0,"num_object_copies":7704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13234,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16196","reported_seq":16430,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.216046+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:34.216046+0000","last_peered":"2024-09-16T04:56:34.216046+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:34.216046+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15912960,"num_objects":3885,"num_object_clones":0,"num_object_copies":7770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12947,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":564}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16441","reported_seq":16577,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.469472+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:34.469472+0000","last_peered":"2024-09-16T04:56:34.469472+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:34.469472+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15470592,"num_objects":3777,"num_object_clones":0,"num_object_copies":7554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13096,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16138","reported_seq":22634,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.195250+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:34.195250+0000","last_peered":"2024-09-16T04:56:34.195250+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:34.195250+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15733970,"num_objects":3843,"num_object_clones":0,"num_object_copies":7686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16110,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3126,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3669}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16195","reported_seq":23057,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.086978+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:34.086978+0000","last_peered":"2024-09-16T04:56:34.086978+0000","last_clean":"2024-09-16T04:56:34.086978+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:34.086978+0000","last_undegraded":"2024-09-16T04:56:34.086978+0000","last_fullsized":"2024-09-16T04:56:34.086978+0000","mapping_epoch":46,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15495168,"num_objects":3783,"num_object_clones":0,"num_object_copies":7566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12918,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16095","reported_seq":16198,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.077896+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:34.077896+0000","last_peered":"2024-09-16T04:56:34.077896+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:34.077896+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15487593,"num_objects":3782,"num_object_clones":0,"num_object_copies":7564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16094,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":375}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16599","reported_seq":16708,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.176350+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:34.176350+0000","last_peered":"2024-09-16T04:56:34.176350+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:34.176350+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16111419,"num_objects":3936,"num_object_clones":0,"num_object_copies":7872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":422}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16309","reported_seq":23109,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:33.727700+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:33.727700+0000","last_peered":"2024-09-16T04:56:33.727700+0000","last_clean":"2024-09-16T04:56:33.727700+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:33.727700+0000","last_undegraded":"2024-09-16T04:56:33.727700+0000","last_fullsized":"2024-09-16T04:56:33.727700+0000","mapping_epoch":48,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15929344,"num_objects":3889,"num_object_clones":0,"num_object_copies":7778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13063,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16541","reported_seq":16687,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.462218+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:34.462218+0000","last_peered":"2024-09-16T04:56:34.462218+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:34.462218+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15872617,"num_objects":3876,"num_object_clones":0,"num_object_copies":7752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13310,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":352}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":250683639,"num_objects":61209,"num_object_clones":0,"num_object_copies":122418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30936,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61209,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225421,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9806,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":668778496,"data_stored":668704251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160943,"ondisk_log_size":160943,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606500,"kb_used_data":236052,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93765340,"statfs":{"total":96636764160,"available":96015708160,"internally_reserved":0,"allocated":241717248,"data_stored":241433922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":379317929},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593812,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550548,"kb_used_data":185668,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93821292,"statfs":{"total":96636764160,"available":96073003008,"internally_reserved":0,"allocated":190124032,"data_stored":189785042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":373616302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617836,"kb_used_data":234968,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93754004,"statfs":{"total":96636764160,"available":96004100096,"internally_reserved":0,"allocated":240607232,"data_stored":240327978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,2,4,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":239108096,"data_stored":239104617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188715008,"data_stored":188651793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":240955392,"data_stored":240947841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:38.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:56:38.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'16011", 'reported_seq': 16096, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:34.451963+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:56:34.451963+0000', 'last_peered': '2024-09-16T04:56:34.451963+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:56:34.451963+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'6000", 'ondisk_log_start': "45'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15265792, 'num_objects': 3727, 'num_object_clones': 0, 'num_object_copies': 7454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3727, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16011, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 386}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:38.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:56:38.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'16100", 'reported_seq': 16245, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:34.192921+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:56:34.192921+0000', 'last_peered': '2024-09-16T04:56:34.192921+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:56:34.192921+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'6000", 'ondisk_log_start': "45'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15388696, 'num_objects': 3758, 'num_object_clones': 0, 'num_object_copies': 7516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3758, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12839, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 439}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:38.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:56:38.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'16460", 'reported_seq': 16597, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:34.502043+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:56:34.502043+0000', 'last_peered': '2024-09-16T04:56:34.502043+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:56:34.502043+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'6400", 'ondisk_log_start': "45'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15781888, 'num_objects': 3853, 'num_object_clones': 0, 'num_object_copies': 7706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13453, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 397}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:38.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:56:38.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'16092", 'reported_seq': 16246, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:34.461174+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:56:34.461174+0000', 'last_peered': '2024-09-16T04:56:34.461174+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:56:34.461174+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'6000", 'ondisk_log_start': "45'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15433728, 'num_objects': 3768, 'num_object_clones': 0, 'num_object_copies': 7536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3768, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12747, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 369}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:38.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:56:38.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "49'16196", 'reported_seq': 16430, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:34.216046+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:56:34.216046+0000', 'last_peered': '2024-09-16T04:56:34.216046+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:56:34.216046+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'6100", 'ondisk_log_start': "45'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15912960, 'num_objects': 3885, 'num_object_clones': 0, 'num_object_copies': 7770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3885, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12947, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 564}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:38.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:56:38.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'16441", 'reported_seq': 16577, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:34.469472+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:56:34.469472+0000', 'last_peered': '2024-09-16T04:56:34.469472+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:56:34.469472+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'6400", 'ondisk_log_start': "45'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15470592, 'num_objects': 3777, 'num_object_clones': 0, 'num_object_copies': 7554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13096, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 312}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:38.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-16T04:56:38.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'16138", 'reported_seq': 22634, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:34.195250+0000', 'last_change': '2024-09-16T04:54:04.130090+0000', 'last_active': '2024-09-16T04:56:34.195250+0000', 'last_peered': '2024-09-16T04:56:34.195250+0000', 'last_clean': '2024-09-16T04:49:35.596121+0000', 'last_became_active': '2024-09-16T04:49:38.104853+0000', 'last_became_peered': '2024-09-16T04:49:38.104853+0000', 'last_unstale': '2024-09-16T04:56:34.195250+0000', 'last_undegraded': '2024-09-16T04:49:37.782505+0000', 'last_fullsized': '2024-09-16T04:49:37.645195+0000', 'mapping_epoch': 44, 'log_start': "45'6100", 'ondisk_log_start': "45'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 3199, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15733970, 'num_objects': 3843, 'num_object_clones': 0, 'num_object_copies': 7686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3843, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16110, 'num_write_kb': 39899, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3126, 'num_bytes_recovered': 3981312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3669}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:38.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:56:38.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'16095", 'reported_seq': 16198, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:34.077896+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:56:34.077896+0000', 'last_peered': '2024-09-16T04:56:34.077896+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:56:34.077896+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'6000", 'ondisk_log_start': "45'6000", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15487593, 'num_objects': 3782, 'num_object_clones': 0, 'num_object_copies': 7564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3782, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16094, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 375}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:38.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:56:38.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "49'16599", 'reported_seq': 16708, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:34.176350+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:56:34.176350+0000', 'last_peered': '2024-09-16T04:56:34.176350+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:56:34.176350+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'6500", 'ondisk_log_start': "45'6500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16111419, 'num_objects': 3936, 'num_object_clones': 0, 'num_object_copies': 7872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3936, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16596, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 422}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:38.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:56:38.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'16541", 'reported_seq': 16687, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:34.462218+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:56:34.462218+0000', 'last_peered': '2024-09-16T04:56:34.462218+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:56:34.462218+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'6500", 'ondisk_log_start': "45'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15872617, 'num_objects': 3876, 'num_object_clones': 0, 'num_object_copies': 7752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3876, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13310, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 352}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:38.276 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:56:39.262 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:39.262 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30936/122422 objects degraded (25.270%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":547,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":61211,"data_bytes":250683658,"bytes_used":1817481216,"bytes_avail":288092811264,"bytes_total":289910292480,"degraded_objects":30936,"degraded_total":122422,"degraded_ratio":0.25269967816242178,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":387},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:56:39.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:56:39.321 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:40.138 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-16T04:56:38.466686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":250683658,"num_objects":61211,"num_object_clones":0,"num_object_copies":122422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30936,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61211,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225423,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9807,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160946,"ondisk_log_size":160946,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1774884,"kb_used_data":656688,"kb_used_omap":12,"kb_used_meta":1118131,"kb_avail":281340636,"statfs":{"total":289910292480,"available":288092811264,"internally_reserved":0,"allocated":672448512,"data_stored":671546942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1144966145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,5,2,3,8,6,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19075072,"num_objects":-4657,"num_object_clones":0,"num_object_copies":-9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4657,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010444"},"pg_stats":[{"pgid":"2.f","version":"49'16011","reported_seq":16096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.451963+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:34.451963+0000","last_peered":"2024-09-16T04:56:34.451963+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:34.451963+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15265792,"num_objects":3727,"num_object_clones":0,"num_object_copies":7454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16011,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":386}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16246","reported_seq":16368,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.183340+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:34.183340+0000","last_peered":"2024-09-16T04:56:34.183340+0000","last_clean":"2024-09-16T04:56:34.183340+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:34.183340+0000","last_undegraded":"2024-09-16T04:56:34.183340+0000","last_fullsized":"2024-09-16T04:56:34.183340+0000","mapping_epoch":35,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15351808,"num_objects":3748,"num_object_clones":0,"num_object_copies":7496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13215,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16100","reported_seq":16245,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.192921+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:34.192921+0000","last_peered":"2024-09-16T04:56:34.192921+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:34.192921+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15388696,"num_objects":3758,"num_object_clones":0,"num_object_copies":7516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12839,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":439}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'16410","reported_seq":16486,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.179860+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:34.179860+0000","last_peered":"2024-09-16T04:56:34.179860+0000","last_clean":"2024-09-16T04:56:34.179860+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:34.179860+0000","last_undegraded":"2024-09-16T04:56:34.179860+0000","last_fullsized":"2024-09-16T04:56:34.179860+0000","mapping_epoch":35,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16113664,"num_objects":3934,"num_object_clones":0,"num_object_copies":7868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16410,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16346","reported_seq":16569,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.475159+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:34.475159+0000","last_peered":"2024-09-16T04:56:34.475159+0000","last_clean":"2024-09-16T04:56:34.475159+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:34.475159+0000","last_undegraded":"2024-09-16T04:56:34.475159+0000","last_fullsized":"2024-09-16T04:56:34.475159+0000","mapping_epoch":34,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15556608,"num_objects":3798,"num_object_clones":0,"num_object_copies":7596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13378,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16460","reported_seq":16597,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.502043+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:34.502043+0000","last_peered":"2024-09-16T04:56:34.502043+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:34.502043+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15781888,"num_objects":3853,"num_object_clones":0,"num_object_copies":7706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13453,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":397}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16092","reported_seq":16246,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.461174+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:34.461174+0000","last_peered":"2024-09-16T04:56:34.461174+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:34.461174+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15433728,"num_objects":3768,"num_object_clones":0,"num_object_copies":7536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12747,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16164","reported_seq":16386,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.393361+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:34.393361+0000","last_peered":"2024-09-16T04:56:34.393361+0000","last_clean":"2024-09-16T04:56:34.393361+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:34.393361+0000","last_undegraded":"2024-09-16T04:56:34.393361+0000","last_fullsized":"2024-09-16T04:56:34.393361+0000","mapping_epoch":34,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15777792,"num_objects":3852,"num_object_clones":0,"num_object_copies":7704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13234,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16196","reported_seq":16430,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.216046+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:34.216046+0000","last_peered":"2024-09-16T04:56:34.216046+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:34.216046+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15912960,"num_objects":3885,"num_object_clones":0,"num_object_copies":7770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12947,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":564}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16441","reported_seq":16577,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.469472+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:34.469472+0000","last_peered":"2024-09-16T04:56:34.469472+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:34.469472+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15470592,"num_objects":3777,"num_object_clones":0,"num_object_copies":7554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13096,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16138","reported_seq":22634,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.195250+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:34.195250+0000","last_peered":"2024-09-16T04:56:34.195250+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:34.195250+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15733970,"num_objects":3843,"num_object_clones":0,"num_object_copies":7686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16110,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3126,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3669}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16195","reported_seq":23057,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:34.086978+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:34.086978+0000","last_peered":"2024-09-16T04:56:34.086978+0000","last_clean":"2024-09-16T04:56:34.086978+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:34.086978+0000","last_undegraded":"2024-09-16T04:56:34.086978+0000","last_fullsized":"2024-09-16T04:56:34.086978+0000","mapping_epoch":46,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15495168,"num_objects":3783,"num_object_clones":0,"num_object_copies":7566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12918,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16095","reported_seq":16198,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.077896+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:34.077896+0000","last_peered":"2024-09-16T04:56:34.077896+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:34.077896+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6000","ondisk_log_start":"45'6000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15487593,"num_objects":3782,"num_object_clones":0,"num_object_copies":7564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16094,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":375}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16599","reported_seq":16708,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.176350+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:34.176350+0000","last_peered":"2024-09-16T04:56:34.176350+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:34.176350+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16111419,"num_objects":3936,"num_object_clones":0,"num_object_copies":7872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":422}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16309","reported_seq":23109,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:33.727700+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:33.727700+0000","last_peered":"2024-09-16T04:56:33.727700+0000","last_clean":"2024-09-16T04:56:33.727700+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:33.727700+0000","last_undegraded":"2024-09-16T04:56:33.727700+0000","last_fullsized":"2024-09-16T04:56:33.727700+0000","mapping_epoch":48,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15929344,"num_objects":3889,"num_object_clones":0,"num_object_copies":7778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13063,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16541","reported_seq":16687,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:34.462218+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:34.462218+0000","last_peered":"2024-09-16T04:56:34.462218+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:34.462218+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15872617,"num_objects":3876,"num_object_clones":0,"num_object_copies":7752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13310,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":352}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":250683639,"num_objects":61209,"num_object_clones":0,"num_object_copies":122418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30936,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61209,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225421,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9806,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":668778496,"data_stored":668704251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160943,"ondisk_log_size":160943,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606500,"kb_used_data":236052,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93765340,"statfs":{"total":96636764160,"available":96015708160,"internally_reserved":0,"allocated":241717248,"data_stored":241433922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":379317929},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593812,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550548,"kb_used_data":185668,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93821292,"statfs":{"total":96636764160,"available":96073003008,"internally_reserved":0,"allocated":190124032,"data_stored":189785042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":373616302},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617836,"kb_used_data":234968,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93754004,"statfs":{"total":96636764160,"available":96004100096,"internally_reserved":0,"allocated":240607232,"data_stored":240327978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,2,4,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":239108096,"data_stored":239104617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188715008,"data_stored":188651793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":240955392,"data_stored":240947841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:43.141 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:44.110 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:44.111 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:44.196 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-16T04:56:42.709636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":240894218,"num_objects":58821,"num_object_clones":0,"num_object_copies":117642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227813,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9925,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160936,"ondisk_log_size":160936,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1760436,"kb_used_data":636672,"kb_used_omap":12,"kb_used_meta":1123699,"kb_avail":281355084,"statfs":{"total":289910292480,"available":288107606016,"internally_reserved":0,"allocated":651952128,"data_stored":651050558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1150667777},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,11,1,5,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13869056,"num_objects":-3386,"num_object_clones":0,"num_object_copies":-6772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3386,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.243931"},"pg_stats":[{"pgid":"2.f","version":"49'16150","reported_seq":16235,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.433032+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:39.433032+0000","last_peered":"2024-09-16T04:56:39.433032+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:39.433032+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16150,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":247}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16392","reported_seq":16512,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:39.237913+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:39.237913+0000","last_peered":"2024-09-16T04:56:39.237913+0000","last_clean":"2024-09-16T04:56:39.237913+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:39.237913+0000","last_undegraded":"2024-09-16T04:56:39.237913+0000","last_fullsized":"2024-09-16T04:56:39.237913+0000","mapping_epoch":35,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14753792,"num_objects":3602,"num_object_clones":0,"num_object_copies":7204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13361,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16248","reported_seq":16393,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.196920+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:39.196920+0000","last_peered":"2024-09-16T04:56:39.196920+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:39.196920+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14782488,"num_objects":3610,"num_object_clones":0,"num_object_copies":7220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3610,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12987,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'16577","reported_seq":16653,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:39.193582+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:39.193582+0000","last_peered":"2024-09-16T04:56:39.193582+0000","last_clean":"2024-09-16T04:56:39.193582+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:39.193582+0000","last_undegraded":"2024-09-16T04:56:39.193582+0000","last_fullsized":"2024-09-16T04:56:39.193582+0000","mapping_epoch":35,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15429632,"num_objects":3767,"num_object_clones":0,"num_object_copies":7534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16577,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16494","reported_seq":16719,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:39.404613+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:39.404613+0000","last_peered":"2024-09-16T04:56:39.404613+0000","last_clean":"2024-09-16T04:56:39.404613+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:39.404613+0000","last_undegraded":"2024-09-16T04:56:39.404613+0000","last_fullsized":"2024-09-16T04:56:39.404613+0000","mapping_epoch":34,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14950400,"num_objects":3650,"num_object_clones":0,"num_object_copies":7300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13526,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16610","reported_seq":16747,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.504739+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:39.504739+0000","last_peered":"2024-09-16T04:56:39.504739+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:39.504739+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15167488,"num_objects":3703,"num_object_clones":0,"num_object_copies":7406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13603,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":247}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16256","reported_seq":16410,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.484457+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:39.484457+0000","last_peered":"2024-09-16T04:56:39.484457+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:39.484457+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14761984,"num_objects":3604,"num_object_clones":0,"num_object_copies":7208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12911,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":205}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16305","reported_seq":16527,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:39.475016+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:39.475016+0000","last_peered":"2024-09-16T04:56:39.475016+0000","last_clean":"2024-09-16T04:56:39.475016+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:39.475016+0000","last_undegraded":"2024-09-16T04:56:39.475016+0000","last_fullsized":"2024-09-16T04:56:39.475016+0000","mapping_epoch":34,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15200256,"num_objects":3711,"num_object_clones":0,"num_object_copies":7422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13375,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.555950+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:54:05.555950+0000","last_peered":"2024-09-16T04:54:05.555950+0000","last_clean":"2024-09-16T04:54:05.555950+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:54:05.555950+0000","last_undegraded":"2024-09-16T04:54:05.555950+0000","last_fullsized":"2024-09-16T04:54:05.555950+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16343","reported_seq":16575,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.254458+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:39.254458+0000","last_peered":"2024-09-16T04:56:39.254458+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:39.254458+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15310848,"num_objects":3738,"num_object_clones":0,"num_object_copies":7476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13094,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":417}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.579657+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:54:05.579657+0000","last_peered":"2024-09-16T04:54:05.579657+0000","last_clean":"2024-09-16T04:54:05.579657+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:54:05.579657+0000","last_undegraded":"2024-09-16T04:54:05.579657+0000","last_fullsized":"2024-09-16T04:54:05.579657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16591","reported_seq":16724,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.472939+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:39.472939+0000","last_peered":"2024-09-16T04:56:39.472939+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:39.472939+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14856192,"num_objects":3627,"num_object_clones":0,"num_object_copies":7254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13246,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":162}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16278","reported_seq":23011,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.230419+0000","last_change":"2024-09-16T04:54:04.130090+0000","last_active":"2024-09-16T04:56:39.230419+0000","last_peered":"2024-09-16T04:56:39.230419+0000","last_clean":"2024-09-16T04:49:35.596121+0000","last_became_active":"2024-09-16T04:49:38.104853+0000","last_became_peered":"2024-09-16T04:49:38.104853+0000","last_unstale":"2024-09-16T04:56:39.230419+0000","last_undegraded":"2024-09-16T04:49:37.782505+0000","last_fullsized":"2024-09-16T04:49:37.645195+0000","mapping_epoch":44,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15160530,"num_objects":3703,"num_object_clones":0,"num_object_copies":7406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16250,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3244,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3647}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567591+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:54:05.567591+0000","last_peered":"2024-09-16T04:54:05.567591+0000","last_clean":"2024-09-16T04:54:05.567591+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:54:05.567591+0000","last_undegraded":"2024-09-16T04:54:05.567591+0000","last_fullsized":"2024-09-16T04:54:05.567591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16348","reported_seq":23210,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:39.155604+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:39.155604+0000","last_peered":"2024-09-16T04:56:39.155604+0000","last_clean":"2024-09-16T04:56:39.155604+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:39.155604+0000","last_undegraded":"2024-09-16T04:56:39.155604+0000","last_fullsized":"2024-09-16T04:56:39.155604+0000","mapping_epoch":46,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14868480,"num_objects":3630,"num_object_clones":0,"num_object_copies":7260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13071,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16224","reported_seq":16328,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.248367+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:39.248367+0000","last_peered":"2024-09-16T04:56:39.248367+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:39.248367+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14959209,"num_objects":3653,"num_object_clones":0,"num_object_copies":7306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16223,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":246}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567616+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:54:05.567616+0000","last_peered":"2024-09-16T04:54:05.567616+0000","last_clean":"2024-09-16T04:54:05.567616+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:54:05.567616+0000","last_undegraded":"2024-09-16T04:54:05.567616+0000","last_fullsized":"2024-09-16T04:54:05.567616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16753","reported_seq":16862,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.234145+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:39.234145+0000","last_peered":"2024-09-16T04:56:39.234145+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:39.234145+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15480635,"num_objects":3782,"num_object_clones":0,"num_object_copies":7564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16750,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":268}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.582871+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:54:05.582871+0000","last_peered":"2024-09-16T04:54:05.582871+0000","last_clean":"2024-09-16T04:54:05.582871+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:54:05.582871+0000","last_undegraded":"2024-09-16T04:54:05.582871+0000","last_fullsized":"2024-09-16T04:54:05.582871+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16467","reported_seq":23267,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:38.750270+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:38.750270+0000","last_peered":"2024-09-16T04:56:38.750270+0000","last_clean":"2024-09-16T04:56:38.750270+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:38.750270+0000","last_undegraded":"2024-09-16T04:56:38.750270+0000","last_fullsized":"2024-09-16T04:56:38.750270+0000","mapping_epoch":48,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15282176,"num_objects":3731,"num_object_clones":0,"num_object_copies":7462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13221,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567830+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:54:05.567830+0000","last_peered":"2024-09-16T04:54:05.567830+0000","last_clean":"2024-09-16T04:54:05.567830+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:54:05.567830+0000","last_undegraded":"2024-09-16T04:54:05.567830+0000","last_fullsized":"2024-09-16T04:54:05.567830+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.567726+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:54:05.567726+0000","last_peered":"2024-09-16T04:54:05.567726+0000","last_clean":"2024-09-16T04:54:05.567726+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:54:05.567726+0000","last_undegraded":"2024-09-16T04:54:05.567726+0000","last_fullsized":"2024-09-16T04:54:05.567726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16697","reported_seq":16843,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.472883+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:39.472883+0000","last_peered":"2024-09-16T04:56:39.472883+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:39.472883+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15233641,"num_objects":3720,"num_object_clones":0,"num_object_copies":7440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13466,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":240894199,"num_objects":58819,"num_object_clones":0,"num_object_copies":117638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58819,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227811,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9924,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":649211904,"data_stored":649137659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604856,"kb_used_data":230248,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93766984,"statfs":{"total":96636764160,"available":96017391616,"internally_reserved":0,"allocated":235773952,"data_stored":235490626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":383577769},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593813,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553136,"kb_used_data":178524,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93818704,"statfs":{"total":96636764160,"available":96070352896,"internally_reserved":0,"allocated":182808576,"data_stored":182469586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":383577774},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,2,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602444,"kb_used_data":227900,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93769396,"statfs":{"total":96636764160,"available":96019861504,"internally_reserved":0,"allocated":233369600,"data_stored":233090346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":232910848,"data_stored":232907369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":181358592,"data_stored":181295377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":234942464,"data_stored":234934913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:44.198 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:44.966 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:44.967 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:45.040 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-16T04:56:44.467984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":236011786,"num_objects":57629,"num_object_clones":0,"num_object_copies":115258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229005,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9981,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1757192,"kb_used_data":625088,"kb_used_omap":12,"kb_used_meta":1132019,"kb_avail":281358328,"statfs":{"total":289910292480,"available":288110927872,"internally_reserved":0,"allocated":640090112,"data_stored":639167900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1159187457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,7,2,5,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14671872,"num_objects":-3582,"num_object_clones":0,"num_object_copies":-7164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3582,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001783"},"pg_stats":[{"pgid":"2.f","version":"49'16150","reported_seq":16235,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.433032+0000","last_change":"2024-09-16T04:49:38.196847+0000","last_active":"2024-09-16T04:56:39.433032+0000","last_peered":"2024-09-16T04:56:39.433032+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:39.433032+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6100","ondisk_log_start":"45'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14696448,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16150,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":247}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'16524","reported_seq":16650,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.244007+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:44.244007+0000","last_peered":"2024-09-16T04:56:44.244007+0000","last_clean":"2024-09-16T04:56:44.244007+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:44.244007+0000","last_undegraded":"2024-09-16T04:56:44.244007+0000","last_fullsized":"2024-09-16T04:56:44.244007+0000","mapping_epoch":35,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14213120,"num_objects":3470,"num_object_clones":0,"num_object_copies":6940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'16388","reported_seq":16537,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:44.240315+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:44.240315+0000","last_peered":"2024-09-16T04:56:44.240315+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:44.240315+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14209048,"num_objects":3470,"num_object_clones":0,"num_object_copies":6940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3470,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13127,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":151}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'16714","reported_seq":16794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.217959+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:44.217959+0000","last_peered":"2024-09-16T04:56:44.217959+0000","last_clean":"2024-09-16T04:56:44.217959+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:44.217959+0000","last_undegraded":"2024-09-16T04:56:44.217959+0000","last_fullsized":"2024-09-16T04:56:44.217959+0000","mapping_epoch":35,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14868480,"num_objects":3630,"num_object_clones":0,"num_object_copies":7260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16714,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16494","reported_seq":16719,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:39.404613+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:39.404613+0000","last_peered":"2024-09-16T04:56:39.404613+0000","last_clean":"2024-09-16T04:56:39.404613+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:39.404613+0000","last_undegraded":"2024-09-16T04:56:39.404613+0000","last_fullsized":"2024-09-16T04:56:39.404613+0000","mapping_epoch":34,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14950400,"num_objects":3650,"num_object_clones":0,"num_object_copies":7300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13526,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16610","reported_seq":16747,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.504739+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:39.504739+0000","last_peered":"2024-09-16T04:56:39.504739+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:39.504739+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15167488,"num_objects":3703,"num_object_clones":0,"num_object_copies":7406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13603,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":247}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16256","reported_seq":16410,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.484457+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:39.484457+0000","last_peered":"2024-09-16T04:56:39.484457+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:39.484457+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14761984,"num_objects":3604,"num_object_clones":0,"num_object_copies":7208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12911,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":205}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16305","reported_seq":16527,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:56:39.475016+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:39.475016+0000","last_peered":"2024-09-16T04:56:39.475016+0000","last_clean":"2024-09-16T04:56:39.475016+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:39.475016+0000","last_undegraded":"2024-09-16T04:56:39.475016+0000","last_fullsized":"2024-09-16T04:56:39.475016+0000","mapping_epoch":34,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15200256,"num_objects":3711,"num_object_clones":0,"num_object_copies":7422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13375,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'16484","reported_seq":16722,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:44.229253+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:44.229253+0000","last_peered":"2024-09-16T04:56:44.229253+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:44.229253+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14733312,"num_objects":3597,"num_object_clones":0,"num_object_copies":7194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13235,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715089+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715089+0000","last_peered":"2024-09-16T04:56:42.715089+0000","last_clean":"2024-09-16T04:56:42.715089+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715089+0000","last_undegraded":"2024-09-16T04:56:42.715089+0000","last_fullsized":"2024-09-16T04:56:42.715089+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16591","reported_seq":16724,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.472939+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:39.472939+0000","last_peered":"2024-09-16T04:56:39.472939+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:39.472939+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14856192,"num_objects":3627,"num_object_clones":0,"num_object_copies":7254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13246,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":162}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T04:54:05.556708+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:54:05.556708+0000","last_peered":"2024-09-16T04:54:05.556708+0000","last_clean":"2024-09-16T04:54:05.556708+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:54:05.556708+0000","last_undegraded":"2024-09-16T04:54:05.556708+0000","last_fullsized":"2024-09-16T04:54:05.556708+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'16417","reported_seq":23272,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.053344+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:56:44.053344+0000","last_peered":"2024-09-16T04:56:44.053344+0000","last_clean":"2024-09-16T04:56:44.053344+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:56:44.053344+0000","last_undegraded":"2024-09-16T04:56:44.053344+0000","last_fullsized":"2024-09-16T04:56:44.053344+0000","mapping_epoch":50,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:47.970877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14591186,"num_objects":3564,"num_object_clones":0,"num_object_copies":7128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16389,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'16482","reported_seq":23348,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.210669+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:44.210669+0000","last_peered":"2024-09-16T04:56:44.210669+0000","last_clean":"2024-09-16T04:56:44.210669+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:44.210669+0000","last_undegraded":"2024-09-16T04:56:44.210669+0000","last_fullsized":"2024-09-16T04:56:44.210669+0000","mapping_epoch":46,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14319616,"num_objects":3496,"num_object_clones":0,"num_object_copies":6992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13205,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'16322","reported_seq":16430,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:44.224885+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:44.224885+0000","last_peered":"2024-09-16T04:56:44.224885+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:44.224885+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14557801,"num_objects":3555,"num_object_clones":0,"num_object_copies":7110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16321,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":148}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'16894","reported_seq":17007,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:44.186388+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:44.186388+0000","last_peered":"2024-09-16T04:56:44.186388+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:44.186388+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14903099,"num_objects":3641,"num_object_clones":0,"num_object_copies":7282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16891,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":127}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715406+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715406+0000","last_peered":"2024-09-16T04:56:42.715406+0000","last_clean":"2024-09-16T04:56:42.715406+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715406+0000","last_undegraded":"2024-09-16T04:56:42.715406+0000","last_fullsized":"2024-09-16T04:56:42.715406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'16597","reported_seq":23401,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:43.798759+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:43.798759+0000","last_peered":"2024-09-16T04:56:43.798759+0000","last_clean":"2024-09-16T04:56:43.798759+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:43.798759+0000","last_undegraded":"2024-09-16T04:56:43.798759+0000","last_fullsized":"2024-09-16T04:56:43.798759+0000","mapping_epoch":48,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14749696,"num_objects":3601,"num_object_clones":0,"num_object_copies":7202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13351,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16697","reported_seq":16843,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:39.472883+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:39.472883+0000","last_peered":"2024-09-16T04:56:39.472883+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:39.472883+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15233641,"num_objects":3720,"num_object_clones":0,"num_object_copies":7440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13466,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":196}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":236011767,"num_objects":57627,"num_object_clones":0,"num_object_copies":115254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57627,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229003,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9980,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":637317120,"data_stored":637242875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160925,"ondisk_log_size":160925,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603628,"kb_used_data":224848,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93768212,"statfs":{"total":96636764160,"available":96018649088,"internally_reserved":0,"allocated":230244352,"data_stored":229950705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":387837609},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593814,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551120,"kb_used_data":172340,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93820720,"statfs":{"total":96636764160,"available":96072417280,"internally_reserved":0,"allocated":176476160,"data_stored":176126849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":387837614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602444,"kb_used_data":227900,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93769396,"statfs":{"total":96636764160,"available":96019861504,"internally_reserved":0,"allocated":233369600,"data_stored":233090346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":232910848,"data_stored":232907369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":174948352,"data_stored":174885137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":229457920,"data_stored":229450369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:45.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:56:45.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'16150", 'reported_seq': 16235, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:39.433032+0000', 'last_change': '2024-09-16T04:49:38.196847+0000', 'last_active': '2024-09-16T04:56:39.433032+0000', 'last_peered': '2024-09-16T04:56:39.433032+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:56:39.433032+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'6100", 'ondisk_log_start': "45'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14696448, 'num_objects': 3588, 'num_object_clones': 0, 'num_object_copies': 7176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3341, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16150, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 247}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:45.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:56:45.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'16388", 'reported_seq': 16537, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:44.240315+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:56:44.240315+0000', 'last_peered': '2024-09-16T04:56:44.240315+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:56:44.240315+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'6300", 'ondisk_log_start': "45'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14209048, 'num_objects': 3470, 'num_object_clones': 0, 'num_object_copies': 6940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3470, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13127, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 151}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:45.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:56:45.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'16610", 'reported_seq': 16747, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:39.504739+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:56:39.504739+0000', 'last_peered': '2024-09-16T04:56:39.504739+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:56:39.504739+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'6600", 'ondisk_log_start': "45'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15167488, 'num_objects': 3703, 'num_object_clones': 0, 'num_object_copies': 7406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3703, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13603, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 247}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:45.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:56:45.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'16256", 'reported_seq': 16410, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:39.484457+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:56:39.484457+0000', 'last_peered': '2024-09-16T04:56:39.484457+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:56:39.484457+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'6200", 'ondisk_log_start': "45'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14761984, 'num_objects': 3604, 'num_object_clones': 0, 'num_object_copies': 7208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3604, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12911, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:45.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:56:45.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'16484", 'reported_seq': 16722, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:44.229253+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:56:44.229253+0000', 'last_peered': '2024-09-16T04:56:44.229253+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:56:44.229253+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'6400", 'ondisk_log_start': "45'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14733312, 'num_objects': 3597, 'num_object_clones': 0, 'num_object_copies': 7194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3597, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13235, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 276}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:45.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:56:45.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'16591", 'reported_seq': 16724, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:39.472939+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:56:39.472939+0000', 'last_peered': '2024-09-16T04:56:39.472939+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:56:39.472939+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'6500", 'ondisk_log_start': "45'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14856192, 'num_objects': 3627, 'num_object_clones': 0, 'num_object_copies': 7254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3627, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13246, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 162}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:45.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:56:45.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'16322", 'reported_seq': 16430, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:44.224885+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:56:44.224885+0000', 'last_peered': '2024-09-16T04:56:44.224885+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:56:44.224885+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'6300", 'ondisk_log_start': "45'6300", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14557801, 'num_objects': 3555, 'num_object_clones': 0, 'num_object_copies': 7110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3555, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16321, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:45.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:56:45.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'16894", 'reported_seq': 17007, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:44.186388+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:56:44.186388+0000', 'last_peered': '2024-09-16T04:56:44.186388+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:56:44.186388+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'6800", 'ondisk_log_start': "45'6800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14903099, 'num_objects': 3641, 'num_object_clones': 0, 'num_object_copies': 7282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3641, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16891, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 127}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:45.043 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:56:45.043 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'16697", 'reported_seq': 16843, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:39.472883+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:56:39.472883+0000', 'last_peered': '2024-09-16T04:56:39.472883+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:56:39.472883+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'6600", 'ondisk_log_start': "45'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15233641, 'num_objects': 3720, 'num_object_clones': 0, 'num_object_copies': 7440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3720, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13466, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 196}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:45.043 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:56:46.022 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:46.022 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30762/115258 objects degraded (26.690%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":553,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9}],"num_pgs":24,"num_pools":2,"num_objects":57629,"data_bytes":236011786,"bytes_used":1799364608,"bytes_avail":288110927872,"bytes_total":289910292480,"degraded_objects":30762,"degraded_total":115258,"degraded_ratio":0.2668968748373215,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":174,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":447},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:56:46.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:56:46.049 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:46.869 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:46.872 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:46.935 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-16T04:56:46.468424+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":232362250,"num_objects":56738,"num_object_clones":0,"num_object_copies":113476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229896,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10088,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160919,"ondisk_log_size":160919,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1756300,"kb_used_data":620028,"kb_used_omap":12,"kb_used_meta":1136179,"kb_avail":281359220,"statfs":{"total":289910292480,"available":288111841280,"internally_reserved":0,"allocated":634908672,"data_stored":633976139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1163447297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18321408,"num_objects":-4473,"num_object_clones":0,"num_object_copies":-8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4473,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001738"},"pg_stats":[{"pgid":"2.f","version":"51'16271","reported_seq":16575,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:44.499000+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:56:44.499000+0000","last_peered":"2024-09-16T04:56:44.499000+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:44.499000+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6200","ondisk_log_start":"45'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14200832,"num_objects":3467,"num_object_clones":0,"num_object_copies":6934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16271,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":233}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'16524","reported_seq":16650,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.244007+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:44.244007+0000","last_peered":"2024-09-16T04:56:44.244007+0000","last_clean":"2024-09-16T04:56:44.244007+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:44.244007+0000","last_undegraded":"2024-09-16T04:56:44.244007+0000","last_fullsized":"2024-09-16T04:56:44.244007+0000","mapping_epoch":35,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14213120,"num_objects":3470,"num_object_clones":0,"num_object_copies":6940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'16388","reported_seq":16537,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:44.240315+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:44.240315+0000","last_peered":"2024-09-16T04:56:44.240315+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:44.240315+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14209048,"num_objects":3470,"num_object_clones":0,"num_object_copies":6940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3470,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13127,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":151}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'16714","reported_seq":16794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.217959+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:44.217959+0000","last_peered":"2024-09-16T04:56:44.217959+0000","last_clean":"2024-09-16T04:56:44.217959+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:44.217959+0000","last_undegraded":"2024-09-16T04:56:44.217959+0000","last_fullsized":"2024-09-16T04:56:44.217959+0000","mapping_epoch":35,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14868480,"num_objects":3630,"num_object_clones":0,"num_object_copies":7260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16714,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'16622","reported_seq":16851,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.473073+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:44.473073+0000","last_peered":"2024-09-16T04:56:44.473073+0000","last_clean":"2024-09-16T04:56:44.473073+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:44.473073+0000","last_undegraded":"2024-09-16T04:56:44.473073+0000","last_fullsized":"2024-09-16T04:56:44.473073+0000","mapping_epoch":34,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14426112,"num_objects":3522,"num_object_clones":0,"num_object_copies":7044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13654,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'16738","reported_seq":16881,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:44.517700+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:44.517700+0000","last_peered":"2024-09-16T04:56:44.517700+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:44.517700+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14643200,"num_objects":3575,"num_object_clones":0,"num_object_copies":7150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13731,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":119}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'16392","reported_seq":16550,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:44.496229+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:44.496229+0000","last_peered":"2024-09-16T04:56:44.496229+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:44.496229+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14204928,"num_objects":3468,"num_object_clones":0,"num_object_copies":6936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13047,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":69}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'16445","reported_seq":16671,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.489766+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:44.489766+0000","last_peered":"2024-09-16T04:56:44.489766+0000","last_clean":"2024-09-16T04:56:44.489766+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:44.489766+0000","last_undegraded":"2024-09-16T04:56:44.489766+0000","last_fullsized":"2024-09-16T04:56:44.489766+0000","mapping_epoch":34,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14626816,"num_objects":3571,"num_object_clones":0,"num_object_copies":7142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13515,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'16484","reported_seq":16722,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:44.229253+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:44.229253+0000","last_peered":"2024-09-16T04:56:44.229253+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:44.229253+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14733312,"num_objects":3597,"num_object_clones":0,"num_object_copies":7194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13235,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715089+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715089+0000","last_peered":"2024-09-16T04:56:42.715089+0000","last_clean":"2024-09-16T04:56:42.715089+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715089+0000","last_undegraded":"2024-09-16T04:56:42.715089+0000","last_fullsized":"2024-09-16T04:56:42.715089+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'16697","reported_seq":16837,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:44.536763+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:44.536763+0000","last_peered":"2024-09-16T04:56:44.536763+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:44.536763+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14422016,"num_objects":3521,"num_object_clones":0,"num_object_copies":7042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13352,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":56}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'16417","reported_seq":23272,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.053344+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:56:44.053344+0000","last_peered":"2024-09-16T04:56:44.053344+0000","last_clean":"2024-09-16T04:56:44.053344+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:56:44.053344+0000","last_undegraded":"2024-09-16T04:56:44.053344+0000","last_fullsized":"2024-09-16T04:56:44.053344+0000","mapping_epoch":50,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:47.970877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14591186,"num_objects":3564,"num_object_clones":0,"num_object_copies":7128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16389,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'16482","reported_seq":23348,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.210669+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:44.210669+0000","last_peered":"2024-09-16T04:56:44.210669+0000","last_clean":"2024-09-16T04:56:44.210669+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:44.210669+0000","last_undegraded":"2024-09-16T04:56:44.210669+0000","last_fullsized":"2024-09-16T04:56:44.210669+0000","mapping_epoch":46,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14319616,"num_objects":3496,"num_object_clones":0,"num_object_copies":6992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13205,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'16322","reported_seq":16430,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:44.224885+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:44.224885+0000","last_peered":"2024-09-16T04:56:44.224885+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:44.224885+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6300","ondisk_log_start":"45'6300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14557801,"num_objects":3555,"num_object_clones":0,"num_object_copies":7110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16321,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":148}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'16894","reported_seq":17007,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:44.186388+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:44.186388+0000","last_peered":"2024-09-16T04:56:44.186388+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:44.186388+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14903099,"num_objects":3641,"num_object_clones":0,"num_object_copies":7282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16891,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":127}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715406+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715406+0000","last_peered":"2024-09-16T04:56:42.715406+0000","last_clean":"2024-09-16T04:56:42.715406+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715406+0000","last_undegraded":"2024-09-16T04:56:42.715406+0000","last_fullsized":"2024-09-16T04:56:42.715406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'16597","reported_seq":23401,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:43.798759+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:43.798759+0000","last_peered":"2024-09-16T04:56:43.798759+0000","last_clean":"2024-09-16T04:56:43.798759+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:43.798759+0000","last_undegraded":"2024-09-16T04:56:43.798759+0000","last_fullsized":"2024-09-16T04:56:43.798759+0000","mapping_epoch":48,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14749696,"num_objects":3601,"num_object_clones":0,"num_object_copies":7202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13351,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'16829","reported_seq":16978,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:44.452176+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:44.452176+0000","last_peered":"2024-09-16T04:56:44.452176+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:44.452176+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14692969,"num_objects":3588,"num_object_clones":0,"num_object_copies":7176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13598,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":64}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":232362231,"num_objects":56736,"num_object_clones":0,"num_object_copies":113472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30655,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56736,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":229894,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10087,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":632074240,"data_stored":631999995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603628,"kb_used_data":224848,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93768212,"statfs":{"total":96636764160,"available":96018649088,"internally_reserved":0,"allocated":230244352,"data_stored":229950705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":387837609},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593814,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551120,"kb_used_data":172340,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93820720,"statfs":{"total":96636764160,"available":96072417280,"internally_reserved":0,"allocated":176476160,"data_stored":176126849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":387837614},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601552,"kb_used_data":222840,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93770288,"statfs":{"total":96636764160,"available":96020774912,"internally_reserved":0,"allocated":228188160,"data_stored":227898585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":227667968,"data_stored":227664489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":174948352,"data_stored":174885137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":229457920,"data_stored":229450369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:49.938 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:50.646 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:50.647 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:50.707 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-16T04:56:50.469291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":222716170,"num_objects":54383,"num_object_clones":0,"num_object_copies":108766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54383,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232251,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10158,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160774,"ondisk_log_size":160774,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1755836,"kb_used_data":601536,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281359684,"statfs":{"total":289910292480,"available":288112316416,"internally_reserved":0,"allocated":615972864,"data_stored":615040331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1181928449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,6,12,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18178048,"num_objects":-4438,"num_object_clones":0,"num_object_copies":-8876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4438,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.767942"},"pg_stats":[{"pgid":"2.f","version":"51'16411","reported_seq":16855,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.513672+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:56:49.513672+0000","last_peered":"2024-09-16T04:56:49.513672+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:49.513672+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13627392,"num_objects":3327,"num_object_clones":0,"num_object_copies":6654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16411,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":163}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'16671","reported_seq":16797,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.244006+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:49.244006+0000","last_peered":"2024-09-16T04:56:49.244006+0000","last_clean":"2024-09-16T04:56:49.244006+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:49.244006+0000","last_undegraded":"2024-09-16T04:56:49.244006+0000","last_fullsized":"2024-09-16T04:56:49.244006+0000","mapping_epoch":35,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13611008,"num_objects":3323,"num_object_clones":0,"num_object_copies":6646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13640,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'16531","reported_seq":16680,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.231723+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:49.231723+0000","last_peered":"2024-09-16T04:56:49.231723+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:49.231723+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13623320,"num_objects":3327,"num_object_clones":0,"num_object_copies":6654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13270,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'16853","reported_seq":16932,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.253499+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:49.253499+0000","last_peered":"2024-09-16T04:56:49.253499+0000","last_clean":"2024-09-16T04:56:49.253499+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:49.253499+0000","last_undegraded":"2024-09-16T04:56:49.253499+0000","last_fullsized":"2024-09-16T04:56:49.253499+0000","mapping_epoch":35,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14299136,"num_objects":3491,"num_object_clones":0,"num_object_copies":6982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16853,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'16777","reported_seq":17006,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.400140+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:49.400140+0000","last_peered":"2024-09-16T04:56:49.400140+0000","last_clean":"2024-09-16T04:56:49.400140+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:49.400140+0000","last_undegraded":"2024-09-16T04:56:49.400140+0000","last_fullsized":"2024-09-16T04:56:49.400140+0000","mapping_epoch":34,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13791232,"num_objects":3367,"num_object_clones":0,"num_object_copies":6734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13809,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'16882","reported_seq":17025,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.412452+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:49.412452+0000","last_peered":"2024-09-16T04:56:49.412452+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:49.412452+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14053376,"num_objects":3431,"num_object_clones":0,"num_object_copies":6862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13875,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-25}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'16532","reported_seq":16690,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.474540+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:49.474540+0000","last_peered":"2024-09-16T04:56:49.474540+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:49.474540+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":3328,"num_object_clones":0,"num_object_copies":6656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13187,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-71}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'16614","reported_seq":16840,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.514842+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:49.514842+0000","last_peered":"2024-09-16T04:56:49.514842+0000","last_clean":"2024-09-16T04:56:49.514842+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:49.514842+0000","last_undegraded":"2024-09-16T04:56:49.514842+0000","last_fullsized":"2024-09-16T04:56:49.514842+0000","mapping_epoch":34,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13934592,"num_objects":3402,"num_object_clones":0,"num_object_copies":6804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13684,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'16621","reported_seq":16859,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.247179+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:49.247179+0000","last_peered":"2024-09-16T04:56:49.247179+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:49.247179+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14172160,"num_objects":3460,"num_object_clones":0,"num_object_copies":6920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13372,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":139}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715089+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715089+0000","last_peered":"2024-09-16T04:56:42.715089+0000","last_clean":"2024-09-16T04:56:42.715089+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715089+0000","last_undegraded":"2024-09-16T04:56:42.715089+0000","last_fullsized":"2024-09-16T04:56:42.715089+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'16838","reported_seq":16977,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.467730+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:49.467730+0000","last_peered":"2024-09-16T04:56:49.467730+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:49.467730+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13844480,"num_objects":3380,"num_object_clones":0,"num_object_copies":6760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-85}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'16551","reported_seq":23406,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.249514+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:56:49.249514+0000","last_peered":"2024-09-16T04:56:49.249514+0000","last_clean":"2024-09-16T04:56:49.249514+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:56:49.249514+0000","last_undegraded":"2024-09-16T04:56:49.249514+0000","last_fullsized":"2024-09-16T04:56:49.249514+0000","mapping_epoch":50,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:53.895232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14042322,"num_objects":3430,"num_object_clones":0,"num_object_copies":6860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16523,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'16613","reported_seq":23480,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.255319+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:49.255319+0000","last_peered":"2024-09-16T04:56:49.255319+0000","last_clean":"2024-09-16T04:56:49.255319+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:49.255319+0000","last_undegraded":"2024-09-16T04:56:49.255319+0000","last_fullsized":"2024-09-16T04:56:49.255319+0000","mapping_epoch":46,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13783040,"num_objects":3365,"num_object_clones":0,"num_object_copies":6730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13336,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'16479","reported_seq":16583,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.267670+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:49.267670+0000","last_peered":"2024-09-16T04:56:49.267670+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:49.267670+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13914729,"num_objects":3398,"num_object_clones":0,"num_object_copies":6796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16478,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-9}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17064","reported_seq":17177,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.182600+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:49.182600+0000","last_peered":"2024-09-16T04:56:49.182600+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:49.182600+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14206779,"num_objects":3471,"num_object_clones":0,"num_object_copies":6942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17061,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-43}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715406+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715406+0000","last_peered":"2024-09-16T04:56:42.715406+0000","last_clean":"2024-09-16T04:56:42.715406+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715406+0000","last_undegraded":"2024-09-16T04:56:42.715406+0000","last_fullsized":"2024-09-16T04:56:42.715406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'16760","reported_seq":23564,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:48.905895+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:48.905895+0000","last_peered":"2024-09-16T04:56:48.905895+0000","last_clean":"2024-09-16T04:56:48.905895+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:48.905895+0000","last_undegraded":"2024-09-16T04:56:48.905895+0000","last_fullsized":"2024-09-16T04:56:48.905895+0000","mapping_epoch":48,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14082048,"num_objects":3438,"num_object_clones":0,"num_object_copies":6876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13514,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'16974","reported_seq":17124,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.508160+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:49.508160+0000","last_peered":"2024-09-16T04:56:49.508160+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:49.508160+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14099049,"num_objects":3443,"num_object_clones":0,"num_object_copies":6886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13743,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-81}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":222716151,"num_objects":54381,"num_object_clones":0,"num_object_copies":108762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232249,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10157,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":612749312,"data_stored":612675067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602036,"kb_used_data":219104,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93769804,"statfs":{"total":96636764160,"available":96020279296,"internally_reserved":0,"allocated":224362496,"data_stored":224068849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":392097449},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593815,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554016,"kb_used_data":165516,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93817824,"statfs":{"total":96636764160,"available":96069451776,"internally_reserved":0,"allocated":169488384,"data_stored":169139073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":397799086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,5,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599784,"kb_used_data":216916,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93772056,"statfs":{"total":96636764160,"available":96022585344,"internally_reserved":0,"allocated":222121984,"data_stored":221832409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":221564928,"data_stored":221561449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":167796736,"data_stored":167733521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":223387648,"data_stored":223380097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:50.708 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:51.536 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:51.611 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-16T04:56:50.469291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":222716170,"num_objects":54383,"num_object_clones":0,"num_object_copies":108766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54383,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232251,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10158,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160774,"ondisk_log_size":160774,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1755836,"kb_used_data":601536,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281359684,"statfs":{"total":289910292480,"available":288112316416,"internally_reserved":0,"allocated":615972864,"data_stored":615040331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1181928449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,6,12,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18178048,"num_objects":-4438,"num_object_clones":0,"num_object_copies":-8876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4438,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.767942"},"pg_stats":[{"pgid":"2.f","version":"51'16411","reported_seq":16855,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.513672+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:56:49.513672+0000","last_peered":"2024-09-16T04:56:49.513672+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:49.513672+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13627392,"num_objects":3327,"num_object_clones":0,"num_object_copies":6654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16411,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":163}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'16671","reported_seq":16797,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.244006+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:49.244006+0000","last_peered":"2024-09-16T04:56:49.244006+0000","last_clean":"2024-09-16T04:56:49.244006+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:49.244006+0000","last_undegraded":"2024-09-16T04:56:49.244006+0000","last_fullsized":"2024-09-16T04:56:49.244006+0000","mapping_epoch":35,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13611008,"num_objects":3323,"num_object_clones":0,"num_object_copies":6646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13640,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'16531","reported_seq":16680,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.231723+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:49.231723+0000","last_peered":"2024-09-16T04:56:49.231723+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:49.231723+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13623320,"num_objects":3327,"num_object_clones":0,"num_object_copies":6654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13270,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'16853","reported_seq":16932,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.253499+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:49.253499+0000","last_peered":"2024-09-16T04:56:49.253499+0000","last_clean":"2024-09-16T04:56:49.253499+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:49.253499+0000","last_undegraded":"2024-09-16T04:56:49.253499+0000","last_fullsized":"2024-09-16T04:56:49.253499+0000","mapping_epoch":35,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14299136,"num_objects":3491,"num_object_clones":0,"num_object_copies":6982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16853,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'16777","reported_seq":17006,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.400140+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:49.400140+0000","last_peered":"2024-09-16T04:56:49.400140+0000","last_clean":"2024-09-16T04:56:49.400140+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:49.400140+0000","last_undegraded":"2024-09-16T04:56:49.400140+0000","last_fullsized":"2024-09-16T04:56:49.400140+0000","mapping_epoch":34,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13791232,"num_objects":3367,"num_object_clones":0,"num_object_copies":6734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13809,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'16882","reported_seq":17025,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.412452+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:49.412452+0000","last_peered":"2024-09-16T04:56:49.412452+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:49.412452+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14053376,"num_objects":3431,"num_object_clones":0,"num_object_copies":6862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13875,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-25}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'16532","reported_seq":16690,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.474540+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:49.474540+0000","last_peered":"2024-09-16T04:56:49.474540+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:49.474540+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":3328,"num_object_clones":0,"num_object_copies":6656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13187,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-71}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'16614","reported_seq":16840,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.514842+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:49.514842+0000","last_peered":"2024-09-16T04:56:49.514842+0000","last_clean":"2024-09-16T04:56:49.514842+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:49.514842+0000","last_undegraded":"2024-09-16T04:56:49.514842+0000","last_fullsized":"2024-09-16T04:56:49.514842+0000","mapping_epoch":34,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13934592,"num_objects":3402,"num_object_clones":0,"num_object_copies":6804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13684,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'16621","reported_seq":16859,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.247179+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:49.247179+0000","last_peered":"2024-09-16T04:56:49.247179+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:49.247179+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14172160,"num_objects":3460,"num_object_clones":0,"num_object_copies":6920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13372,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":139}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715089+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715089+0000","last_peered":"2024-09-16T04:56:42.715089+0000","last_clean":"2024-09-16T04:56:42.715089+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715089+0000","last_undegraded":"2024-09-16T04:56:42.715089+0000","last_fullsized":"2024-09-16T04:56:42.715089+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'16838","reported_seq":16977,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.467730+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:49.467730+0000","last_peered":"2024-09-16T04:56:49.467730+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:49.467730+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13844480,"num_objects":3380,"num_object_clones":0,"num_object_copies":6760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-85}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'16551","reported_seq":23406,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.249514+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:56:49.249514+0000","last_peered":"2024-09-16T04:56:49.249514+0000","last_clean":"2024-09-16T04:56:49.249514+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:56:49.249514+0000","last_undegraded":"2024-09-16T04:56:49.249514+0000","last_fullsized":"2024-09-16T04:56:49.249514+0000","mapping_epoch":50,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:53.895232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14042322,"num_objects":3430,"num_object_clones":0,"num_object_copies":6860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16523,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'16613","reported_seq":23480,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.255319+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:49.255319+0000","last_peered":"2024-09-16T04:56:49.255319+0000","last_clean":"2024-09-16T04:56:49.255319+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:49.255319+0000","last_undegraded":"2024-09-16T04:56:49.255319+0000","last_fullsized":"2024-09-16T04:56:49.255319+0000","mapping_epoch":46,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13783040,"num_objects":3365,"num_object_clones":0,"num_object_copies":6730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13336,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'16479","reported_seq":16583,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.267670+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:49.267670+0000","last_peered":"2024-09-16T04:56:49.267670+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:49.267670+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13914729,"num_objects":3398,"num_object_clones":0,"num_object_copies":6796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16478,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-9}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17064","reported_seq":17177,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.182600+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:49.182600+0000","last_peered":"2024-09-16T04:56:49.182600+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:49.182600+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14206779,"num_objects":3471,"num_object_clones":0,"num_object_copies":6942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17061,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-43}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715406+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715406+0000","last_peered":"2024-09-16T04:56:42.715406+0000","last_clean":"2024-09-16T04:56:42.715406+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715406+0000","last_undegraded":"2024-09-16T04:56:42.715406+0000","last_fullsized":"2024-09-16T04:56:42.715406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'16760","reported_seq":23564,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:48.905895+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:48.905895+0000","last_peered":"2024-09-16T04:56:48.905895+0000","last_clean":"2024-09-16T04:56:48.905895+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:48.905895+0000","last_undegraded":"2024-09-16T04:56:48.905895+0000","last_fullsized":"2024-09-16T04:56:48.905895+0000","mapping_epoch":48,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14082048,"num_objects":3438,"num_object_clones":0,"num_object_copies":6876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13514,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'16974","reported_seq":17124,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.508160+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:49.508160+0000","last_peered":"2024-09-16T04:56:49.508160+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:49.508160+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14099049,"num_objects":3443,"num_object_clones":0,"num_object_copies":6886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13743,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-81}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":222716151,"num_objects":54381,"num_object_clones":0,"num_object_copies":108762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232249,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10157,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":612749312,"data_stored":612675067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602036,"kb_used_data":219104,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93769804,"statfs":{"total":96636764160,"available":96020279296,"internally_reserved":0,"allocated":224362496,"data_stored":224068849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":392097449},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593815,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554016,"kb_used_data":165516,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93817824,"statfs":{"total":96636764160,"available":96069451776,"internally_reserved":0,"allocated":169488384,"data_stored":169139073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":397799086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,5,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599784,"kb_used_data":216916,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93772056,"statfs":{"total":96636764160,"available":96022585344,"internally_reserved":0,"allocated":222121984,"data_stored":221832409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":221564928,"data_stored":221561449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":167796736,"data_stored":167733521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":223387648,"data_stored":223380097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:51.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:56:51.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'16411", 'reported_seq': 16855, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:49.513672+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:56:49.513672+0000', 'last_peered': '2024-09-16T04:56:49.513672+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:56:49.513672+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'6400", 'ondisk_log_start': "45'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13627392, 'num_objects': 3327, 'num_object_clones': 0, 'num_object_copies': 6654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3164, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3327, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16411, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 178, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 163}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:51.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:56:51.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'16531", 'reported_seq': 16680, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:49.231723+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:56:49.231723+0000', 'last_peered': '2024-09-16T04:56:49.231723+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:56:49.231723+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'6500", 'ondisk_log_start': "45'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13623320, 'num_objects': 3327, 'num_object_clones': 0, 'num_object_copies': 6654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3327, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13270, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 8}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:51.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:56:51.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'16882", 'reported_seq': 17025, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:49.412452+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:56:49.412452+0000', 'last_peered': '2024-09-16T04:56:49.412452+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:56:49.412452+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'6800", 'ondisk_log_start': "45'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14053376, 'num_objects': 3431, 'num_object_clones': 0, 'num_object_copies': 6862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3431, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13875, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -25}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:51.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:56:51.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'16532", 'reported_seq': 16690, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:49.474540+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:56:49.474540+0000', 'last_peered': '2024-09-16T04:56:49.474540+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:56:49.474540+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'6500", 'ondisk_log_start': "45'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13631488, 'num_objects': 3328, 'num_object_clones': 0, 'num_object_copies': 6656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3328, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13187, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -71}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:51.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:56:51.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'16621", 'reported_seq': 16859, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:49.247179+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:56:49.247179+0000', 'last_peered': '2024-09-16T04:56:49.247179+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:56:49.247179+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'6600", 'ondisk_log_start': "45'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14172160, 'num_objects': 3460, 'num_object_clones': 0, 'num_object_copies': 6920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3460, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13372, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:51.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:56:51.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'16838", 'reported_seq': 16977, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:49.467730+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:56:49.467730+0000', 'last_peered': '2024-09-16T04:56:49.467730+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:56:49.467730+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'6800", 'ondisk_log_start': "45'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13844480, 'num_objects': 3380, 'num_object_clones': 0, 'num_object_copies': 6760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3380, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13493, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -85}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:51.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:56:51.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'16479", 'reported_seq': 16583, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:49.267670+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:56:49.267670+0000', 'last_peered': '2024-09-16T04:56:49.267670+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:56:49.267670+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'6400", 'ondisk_log_start': "45'6400", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13914729, 'num_objects': 3398, 'num_object_clones': 0, 'num_object_copies': 6796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3398, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16478, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -9}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:51.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:56:51.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'17064", 'reported_seq': 17177, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:49.182600+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:56:49.182600+0000', 'last_peered': '2024-09-16T04:56:49.182600+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:56:49.182600+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'7000", 'ondisk_log_start': "45'7000", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14206779, 'num_objects': 3471, 'num_object_clones': 0, 'num_object_copies': 6942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3471, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17061, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -43}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:51.614 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:56:51.614 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'16974", 'reported_seq': 17124, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:49.508160+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:56:49.508160+0000', 'last_peered': '2024-09-16T04:56:49.508160+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:56:49.508160+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'6900", 'ondisk_log_start': "45'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14099049, 'num_objects': 3443, 'num_object_clones': 0, 'num_object_copies': 6886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3443, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13743, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -81}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:51.614 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:56:52.669 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:52.669 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30585/108766 objects degraded (28.120%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":560,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":54383,"data_bytes":222716170,"bytes_used":1797976064,"bytes_avail":288112316416,"bytes_total":289910292480,"degraded_objects":30585,"degraded_total":108766,"degraded_ratio":0.2812000073552397,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":443},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:56:52.708 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:56:52.709 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:53.578 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:53.578 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:53.633 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-16T04:56:52.469807+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":222716170,"num_objects":54383,"num_object_clones":0,"num_object_copies":108766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54383,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232251,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10158,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160774,"ondisk_log_size":160774,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1755836,"kb_used_data":601536,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281359684,"statfs":{"total":289910292480,"available":288112316416,"internally_reserved":0,"allocated":615972864,"data_stored":615040331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1181928449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,6,12,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18178048,"num_objects":-4438,"num_object_clones":0,"num_object_copies":-8876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-233,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4438,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002195"},"pg_stats":[{"pgid":"2.f","version":"51'16411","reported_seq":16855,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.513672+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:56:49.513672+0000","last_peered":"2024-09-16T04:56:49.513672+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:49.513672+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13627392,"num_objects":3327,"num_object_clones":0,"num_object_copies":6654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16411,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":163}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'16671","reported_seq":16797,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.244006+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:49.244006+0000","last_peered":"2024-09-16T04:56:49.244006+0000","last_clean":"2024-09-16T04:56:49.244006+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:49.244006+0000","last_undegraded":"2024-09-16T04:56:49.244006+0000","last_fullsized":"2024-09-16T04:56:49.244006+0000","mapping_epoch":35,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13611008,"num_objects":3323,"num_object_clones":0,"num_object_copies":6646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13640,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'16531","reported_seq":16680,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.231723+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:49.231723+0000","last_peered":"2024-09-16T04:56:49.231723+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:49.231723+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13623320,"num_objects":3327,"num_object_clones":0,"num_object_copies":6654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13270,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'16853","reported_seq":16932,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.253499+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:49.253499+0000","last_peered":"2024-09-16T04:56:49.253499+0000","last_clean":"2024-09-16T04:56:49.253499+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:49.253499+0000","last_undegraded":"2024-09-16T04:56:49.253499+0000","last_fullsized":"2024-09-16T04:56:49.253499+0000","mapping_epoch":35,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14299136,"num_objects":3491,"num_object_clones":0,"num_object_copies":6982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16853,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'16777","reported_seq":17006,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.400140+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:49.400140+0000","last_peered":"2024-09-16T04:56:49.400140+0000","last_clean":"2024-09-16T04:56:49.400140+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:49.400140+0000","last_undegraded":"2024-09-16T04:56:49.400140+0000","last_fullsized":"2024-09-16T04:56:49.400140+0000","mapping_epoch":34,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13791232,"num_objects":3367,"num_object_clones":0,"num_object_copies":6734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13809,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'16882","reported_seq":17025,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.412452+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:49.412452+0000","last_peered":"2024-09-16T04:56:49.412452+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:49.412452+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14053376,"num_objects":3431,"num_object_clones":0,"num_object_copies":6862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13875,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-25}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'16532","reported_seq":16690,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.474540+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:49.474540+0000","last_peered":"2024-09-16T04:56:49.474540+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:49.474540+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":3328,"num_object_clones":0,"num_object_copies":6656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13187,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-71}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'16614","reported_seq":16840,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.514842+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:49.514842+0000","last_peered":"2024-09-16T04:56:49.514842+0000","last_clean":"2024-09-16T04:56:49.514842+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:49.514842+0000","last_undegraded":"2024-09-16T04:56:49.514842+0000","last_fullsized":"2024-09-16T04:56:49.514842+0000","mapping_epoch":34,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13934592,"num_objects":3402,"num_object_clones":0,"num_object_copies":6804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13684,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'16621","reported_seq":16859,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.247179+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:49.247179+0000","last_peered":"2024-09-16T04:56:49.247179+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:49.247179+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14172160,"num_objects":3460,"num_object_clones":0,"num_object_copies":6920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13372,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":139}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715089+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715089+0000","last_peered":"2024-09-16T04:56:42.715089+0000","last_clean":"2024-09-16T04:56:42.715089+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715089+0000","last_undegraded":"2024-09-16T04:56:42.715089+0000","last_fullsized":"2024-09-16T04:56:42.715089+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'16838","reported_seq":16977,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.467730+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:49.467730+0000","last_peered":"2024-09-16T04:56:49.467730+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:49.467730+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13844480,"num_objects":3380,"num_object_clones":0,"num_object_copies":6760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-85}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'16551","reported_seq":23406,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.249514+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:56:49.249514+0000","last_peered":"2024-09-16T04:56:49.249514+0000","last_clean":"2024-09-16T04:56:49.249514+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:56:49.249514+0000","last_undegraded":"2024-09-16T04:56:49.249514+0000","last_fullsized":"2024-09-16T04:56:49.249514+0000","mapping_epoch":50,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:53.895232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14042322,"num_objects":3430,"num_object_clones":0,"num_object_copies":6860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16523,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'16613","reported_seq":23480,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:49.255319+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:49.255319+0000","last_peered":"2024-09-16T04:56:49.255319+0000","last_clean":"2024-09-16T04:56:49.255319+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:49.255319+0000","last_undegraded":"2024-09-16T04:56:49.255319+0000","last_fullsized":"2024-09-16T04:56:49.255319+0000","mapping_epoch":46,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13783040,"num_objects":3365,"num_object_clones":0,"num_object_copies":6730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13336,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'16479","reported_seq":16583,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.267670+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:49.267670+0000","last_peered":"2024-09-16T04:56:49.267670+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:49.267670+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6400","ondisk_log_start":"45'6400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13914729,"num_objects":3398,"num_object_clones":0,"num_object_copies":6796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16478,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-9}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17064","reported_seq":17177,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.182600+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:49.182600+0000","last_peered":"2024-09-16T04:56:49.182600+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:49.182600+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14206779,"num_objects":3471,"num_object_clones":0,"num_object_copies":6942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17061,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-43}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715406+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715406+0000","last_peered":"2024-09-16T04:56:42.715406+0000","last_clean":"2024-09-16T04:56:42.715406+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715406+0000","last_undegraded":"2024-09-16T04:56:42.715406+0000","last_fullsized":"2024-09-16T04:56:42.715406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'16760","reported_seq":23564,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:48.905895+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:48.905895+0000","last_peered":"2024-09-16T04:56:48.905895+0000","last_clean":"2024-09-16T04:56:48.905895+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:48.905895+0000","last_undegraded":"2024-09-16T04:56:48.905895+0000","last_fullsized":"2024-09-16T04:56:48.905895+0000","mapping_epoch":48,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14082048,"num_objects":3438,"num_object_clones":0,"num_object_copies":6876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13514,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'16974","reported_seq":17124,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:49.508160+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:49.508160+0000","last_peered":"2024-09-16T04:56:49.508160+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:49.508160+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14099049,"num_objects":3443,"num_object_clones":0,"num_object_copies":6886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13743,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-81}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":222716151,"num_objects":54381,"num_object_clones":0,"num_object_copies":108762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232249,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10157,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":612749312,"data_stored":612675067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602036,"kb_used_data":219104,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93769804,"statfs":{"total":96636764160,"available":96020279296,"internally_reserved":0,"allocated":224362496,"data_stored":224068849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":392097449},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593815,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554016,"kb_used_data":165516,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93817824,"statfs":{"total":96636764160,"available":96069451776,"internally_reserved":0,"allocated":169488384,"data_stored":169139073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":397799086},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,5,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599784,"kb_used_data":216916,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93772056,"statfs":{"total":96636764160,"available":96022585344,"internally_reserved":0,"allocated":222121984,"data_stored":221832409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":221564928,"data_stored":221561449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":167796736,"data_stored":167733521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":223387648,"data_stored":223380097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:56.635 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:57.509 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:57.509 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:57.581 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-16T04:56:56.470722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":213160202,"num_objects":52050,"num_object_clones":0,"num_object_copies":104100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234584,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10243,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160907,"ondisk_log_size":160907,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1720444,"kb_used_data":582776,"kb_used_omap":12,"kb_used_meta":1137587,"kb_avail":281395076,"statfs":{"total":289910292480,"available":288148557824,"internally_reserved":0,"allocated":596762624,"data_stored":595830091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1164889089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,1,1,8,12,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22851584,"num_objects":-5579,"num_object_clones":0,"num_object_copies":-11158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5579,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002738"},"pg_stats":[{"pgid":"2.f","version":"51'16550","reported_seq":17164,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.580576+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:56:54.580576+0000","last_peered":"2024-09-16T04:56:54.580576+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:54.580576+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13058048,"num_objects":3188,"num_object_clones":0,"num_object_copies":6376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16550,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":109}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'16827","reported_seq":16953,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.255334+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:54.255334+0000","last_peered":"2024-09-16T04:56:54.255334+0000","last_clean":"2024-09-16T04:56:54.255334+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:54.255334+0000","last_undegraded":"2024-09-16T04:56:54.255334+0000","last_fullsized":"2024-09-16T04:56:54.255334+0000","mapping_epoch":35,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12972032,"num_objects":3167,"num_object_clones":0,"num_object_copies":6334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13796,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'16661","reported_seq":16810,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.275191+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:54.275191+0000","last_peered":"2024-09-16T04:56:54.275191+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:54.275191+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13090840,"num_objects":3197,"num_object_clones":0,"num_object_copies":6394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3197,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13400,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'16997","reported_seq":17076,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.323317+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:54.323317+0000","last_peered":"2024-09-16T04:56:54.323317+0000","last_clean":"2024-09-16T04:56:54.323317+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:54.323317+0000","last_undegraded":"2024-09-16T04:56:54.323317+0000","last_fullsized":"2024-09-16T04:56:54.323317+0000","mapping_epoch":35,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13709312,"num_objects":3347,"num_object_clones":0,"num_object_copies":6694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16997,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'16910","reported_seq":17139,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.580483+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:54.580483+0000","last_peered":"2024-09-16T04:56:54.580483+0000","last_clean":"2024-09-16T04:56:54.580483+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:54.580483+0000","last_undegraded":"2024-09-16T04:56:54.580483+0000","last_fullsized":"2024-09-16T04:56:54.580483+0000","mapping_epoch":34,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13246464,"num_objects":3234,"num_object_clones":0,"num_object_copies":6468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13942,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17036","reported_seq":17179,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.490168+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:54.490168+0000","last_peered":"2024-09-16T04:56:54.490168+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:54.490168+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13422592,"num_objects":3277,"num_object_clones":0,"num_object_copies":6554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14029,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'16669","reported_seq":16827,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.458572+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:54.458572+0000","last_peered":"2024-09-16T04:56:54.458572+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:54.458572+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13070336,"num_objects":3191,"num_object_clones":0,"num_object_copies":6382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13324,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-208}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'16766","reported_seq":16992,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.559721+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:54.559721+0000","last_peered":"2024-09-16T04:56:54.559721+0000","last_clean":"2024-09-16T04:56:54.559721+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:54.559721+0000","last_undegraded":"2024-09-16T04:56:54.559721+0000","last_fullsized":"2024-09-16T04:56:54.559721+0000","mapping_epoch":34,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13312000,"num_objects":3250,"num_object_clones":0,"num_object_copies":6500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13836,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'16785","reported_seq":17023,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.313095+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:54.313095+0000","last_peered":"2024-09-16T04:56:54.313095+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:54.313095+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":3296,"num_object_clones":0,"num_object_copies":6592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13536,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-25}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715089+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715089+0000","last_peered":"2024-09-16T04:56:42.715089+0000","last_clean":"2024-09-16T04:56:42.715089+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715089+0000","last_undegraded":"2024-09-16T04:56:42.715089+0000","last_fullsized":"2024-09-16T04:56:42.715089+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'16998","reported_seq":17136,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.471607+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:54.471607+0000","last_peered":"2024-09-16T04:56:54.471607+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:54.471607+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13189120,"num_objects":3220,"num_object_clones":0,"num_object_copies":6440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13653,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-245}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'16688","reported_seq":23543,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.180425+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:56:54.180425+0000","last_peered":"2024-09-16T04:56:54.180425+0000","last_clean":"2024-09-16T04:56:54.180425+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:56:54.180425+0000","last_undegraded":"2024-09-16T04:56:54.180425+0000","last_fullsized":"2024-09-16T04:56:54.180425+0000","mapping_epoch":50,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:59.009028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13481170,"num_objects":3293,"num_object_clones":0,"num_object_copies":6586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16660,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'16770","reported_seq":23637,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.269131+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:54.269131+0000","last_peered":"2024-09-16T04:56:54.269131+0000","last_clean":"2024-09-16T04:56:54.269131+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:54.269131+0000","last_undegraded":"2024-09-16T04:56:54.269131+0000","last_fullsized":"2024-09-16T04:56:54.269131+0000","mapping_epoch":46,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13139968,"num_objects":3208,"num_object_clones":0,"num_object_copies":6416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'16630","reported_seq":16733,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.263697+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:54.263697+0000","last_peered":"2024-09-16T04:56:54.263697+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:54.263697+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13296233,"num_objects":3247,"num_object_clones":0,"num_object_copies":6494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16629,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-160}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17220","reported_seq":17333,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.297151+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:54.297151+0000","last_peered":"2024-09-16T04:56:54.297151+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:54.297151+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13567803,"num_objects":3315,"num_object_clones":0,"num_object_copies":6630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17217,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-199}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715406+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715406+0000","last_peered":"2024-09-16T04:56:42.715406+0000","last_clean":"2024-09-16T04:56:42.715406+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715406+0000","last_undegraded":"2024-09-16T04:56:42.715406+0000","last_fullsized":"2024-09-16T04:56:42.715406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'16901","reported_seq":23704,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.003235+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:54.003235+0000","last_peered":"2024-09-16T04:56:54.003235+0000","last_clean":"2024-09-16T04:56:54.003235+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:54.003235+0000","last_undegraded":"2024-09-16T04:56:54.003235+0000","last_fullsized":"2024-09-16T04:56:54.003235+0000","mapping_epoch":48,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13504512,"num_objects":3297,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13655,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17096","reported_seq":17246,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.586830+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:54.586830+0000","last_peered":"2024-09-16T04:56:54.586830+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:54.586830+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13599337,"num_objects":3321,"num_object_clones":0,"num_object_copies":6642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13865,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":213160183,"num_objects":52048,"num_object_clones":0,"num_object_copies":104096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234582,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10242,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593588224,"data_stored":593513979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587704,"kb_used_data":213092,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93784136,"statfs":{"total":96636764160,"available":96034955264,"internally_reserved":0,"allocated":218206208,"data_stored":217912561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":383577769},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593816,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551232,"kb_used_data":158564,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93820608,"statfs":{"total":96636764160,"available":96072302592,"internally_reserved":0,"allocated":162369536,"data_stored":162020225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":402058926},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,5,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581508,"kb_used_data":211120,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93790332,"statfs":{"total":96636764160,"available":96041299968,"internally_reserved":0,"allocated":216186880,"data_stored":215897305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":215629824,"data_stored":215626345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":160591872,"data_stored":160528657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217366528,"data_stored":217358977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:57.582 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:56:58.675 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:58.676 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:56:58.757 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-16T04:56:58.472779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":213160202,"num_objects":52050,"num_object_clones":0,"num_object_copies":104100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234584,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10243,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160907,"ondisk_log_size":160907,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1720444,"kb_used_data":582776,"kb_used_omap":12,"kb_used_meta":1137587,"kb_avail":281395076,"statfs":{"total":289910292480,"available":288148557824,"internally_reserved":0,"allocated":596762624,"data_stored":595830091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1164889089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,1,1,8,12,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19202048,"num_objects":-4688,"num_object_clones":0,"num_object_copies":-9376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4688,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004354"},"pg_stats":[{"pgid":"2.f","version":"51'16550","reported_seq":17164,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.580576+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:56:54.580576+0000","last_peered":"2024-09-16T04:56:54.580576+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:54.580576+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13058048,"num_objects":3188,"num_object_clones":0,"num_object_copies":6376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16550,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":109}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'16827","reported_seq":16953,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.255334+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:54.255334+0000","last_peered":"2024-09-16T04:56:54.255334+0000","last_clean":"2024-09-16T04:56:54.255334+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:54.255334+0000","last_undegraded":"2024-09-16T04:56:54.255334+0000","last_fullsized":"2024-09-16T04:56:54.255334+0000","mapping_epoch":35,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12972032,"num_objects":3167,"num_object_clones":0,"num_object_copies":6334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13796,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'16661","reported_seq":16810,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.275191+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:54.275191+0000","last_peered":"2024-09-16T04:56:54.275191+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:54.275191+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13090840,"num_objects":3197,"num_object_clones":0,"num_object_copies":6394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3197,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13400,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'16997","reported_seq":17076,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.323317+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:54.323317+0000","last_peered":"2024-09-16T04:56:54.323317+0000","last_clean":"2024-09-16T04:56:54.323317+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:54.323317+0000","last_undegraded":"2024-09-16T04:56:54.323317+0000","last_fullsized":"2024-09-16T04:56:54.323317+0000","mapping_epoch":35,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13709312,"num_objects":3347,"num_object_clones":0,"num_object_copies":6694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16997,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'16910","reported_seq":17139,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.580483+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:54.580483+0000","last_peered":"2024-09-16T04:56:54.580483+0000","last_clean":"2024-09-16T04:56:54.580483+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:54.580483+0000","last_undegraded":"2024-09-16T04:56:54.580483+0000","last_fullsized":"2024-09-16T04:56:54.580483+0000","mapping_epoch":34,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13246464,"num_objects":3234,"num_object_clones":0,"num_object_copies":6468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13942,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17036","reported_seq":17179,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.490168+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:54.490168+0000","last_peered":"2024-09-16T04:56:54.490168+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:54.490168+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13422592,"num_objects":3277,"num_object_clones":0,"num_object_copies":6554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14029,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'16669","reported_seq":16827,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.458572+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:54.458572+0000","last_peered":"2024-09-16T04:56:54.458572+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:54.458572+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13070336,"num_objects":3191,"num_object_clones":0,"num_object_copies":6382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13324,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-208}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'16766","reported_seq":16992,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.559721+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:54.559721+0000","last_peered":"2024-09-16T04:56:54.559721+0000","last_clean":"2024-09-16T04:56:54.559721+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:54.559721+0000","last_undegraded":"2024-09-16T04:56:54.559721+0000","last_fullsized":"2024-09-16T04:56:54.559721+0000","mapping_epoch":34,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13312000,"num_objects":3250,"num_object_clones":0,"num_object_copies":6500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13836,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'16785","reported_seq":17023,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.313095+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:54.313095+0000","last_peered":"2024-09-16T04:56:54.313095+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:54.313095+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":3296,"num_object_clones":0,"num_object_copies":6592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13536,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-25}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715089+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715089+0000","last_peered":"2024-09-16T04:56:42.715089+0000","last_clean":"2024-09-16T04:56:42.715089+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715089+0000","last_undegraded":"2024-09-16T04:56:42.715089+0000","last_fullsized":"2024-09-16T04:56:42.715089+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'16998","reported_seq":17136,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.471607+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:54.471607+0000","last_peered":"2024-09-16T04:56:54.471607+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:54.471607+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13189120,"num_objects":3220,"num_object_clones":0,"num_object_copies":6440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13653,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-245}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'16688","reported_seq":23543,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.180425+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:56:54.180425+0000","last_peered":"2024-09-16T04:56:54.180425+0000","last_clean":"2024-09-16T04:56:54.180425+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:56:54.180425+0000","last_undegraded":"2024-09-16T04:56:54.180425+0000","last_fullsized":"2024-09-16T04:56:54.180425+0000","mapping_epoch":50,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:59.009028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13481170,"num_objects":3293,"num_object_clones":0,"num_object_copies":6586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16660,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'16770","reported_seq":23637,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.269131+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:54.269131+0000","last_peered":"2024-09-16T04:56:54.269131+0000","last_clean":"2024-09-16T04:56:54.269131+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:54.269131+0000","last_undegraded":"2024-09-16T04:56:54.269131+0000","last_fullsized":"2024-09-16T04:56:54.269131+0000","mapping_epoch":46,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13139968,"num_objects":3208,"num_object_clones":0,"num_object_copies":6416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'16630","reported_seq":16733,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.263697+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:54.263697+0000","last_peered":"2024-09-16T04:56:54.263697+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:54.263697+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13296233,"num_objects":3247,"num_object_clones":0,"num_object_copies":6494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16629,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-160}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17220","reported_seq":17333,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.297151+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:54.297151+0000","last_peered":"2024-09-16T04:56:54.297151+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:54.297151+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13567803,"num_objects":3315,"num_object_clones":0,"num_object_copies":6630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17217,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-199}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715406+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715406+0000","last_peered":"2024-09-16T04:56:42.715406+0000","last_clean":"2024-09-16T04:56:42.715406+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715406+0000","last_undegraded":"2024-09-16T04:56:42.715406+0000","last_fullsized":"2024-09-16T04:56:42.715406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'16901","reported_seq":23704,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.003235+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:54.003235+0000","last_peered":"2024-09-16T04:56:54.003235+0000","last_clean":"2024-09-16T04:56:54.003235+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:54.003235+0000","last_undegraded":"2024-09-16T04:56:54.003235+0000","last_fullsized":"2024-09-16T04:56:54.003235+0000","mapping_epoch":48,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13504512,"num_objects":3297,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13655,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17096","reported_seq":17246,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.586830+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:54.586830+0000","last_peered":"2024-09-16T04:56:54.586830+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:54.586830+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13599337,"num_objects":3321,"num_object_clones":0,"num_object_copies":6642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13865,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":213160183,"num_objects":52048,"num_object_clones":0,"num_object_copies":104096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234582,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10242,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593588224,"data_stored":593513979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587704,"kb_used_data":213092,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93784136,"statfs":{"total":96636764160,"available":96034955264,"internally_reserved":0,"allocated":218206208,"data_stored":217912561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":383577769},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593816,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551232,"kb_used_data":158564,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93820608,"statfs":{"total":96636764160,"available":96072302592,"internally_reserved":0,"allocated":162369536,"data_stored":162020225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":402058926},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,5,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581508,"kb_used_data":211120,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93790332,"statfs":{"total":96636764160,"available":96041299968,"internally_reserved":0,"allocated":216186880,"data_stored":215897305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":215629824,"data_stored":215626345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":160591872,"data_stored":160528657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217366528,"data_stored":217358977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:56:58.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:56:58.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'16550", 'reported_seq': 17164, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:54.580576+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:56:54.580576+0000', 'last_peered': '2024-09-16T04:56:54.580576+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:56:54.580576+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'6500", 'ondisk_log_start': "45'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13058048, 'num_objects': 3188, 'num_object_clones': 0, 'num_object_copies': 6376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3079, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3188, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16550, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 263, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 109}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:58.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:56:58.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'16661", 'reported_seq': 16810, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:54.275191+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:56:54.275191+0000', 'last_peered': '2024-09-16T04:56:54.275191+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:56:54.275191+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'6600", 'ondisk_log_start': "45'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13090840, 'num_objects': 3197, 'num_object_clones': 0, 'num_object_copies': 6394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3197, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13400, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -122}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:58.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:56:58.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'17036", 'reported_seq': 17179, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:54.490168+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:56:54.490168+0000', 'last_peered': '2024-09-16T04:56:54.490168+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:56:54.490168+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'7000", 'ondisk_log_start': "45'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13422592, 'num_objects': 3277, 'num_object_clones': 0, 'num_object_copies': 6554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3277, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14029, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -179}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:58.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:56:58.760 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'16669", 'reported_seq': 16827, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:54.458572+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:56:54.458572+0000', 'last_peered': '2024-09-16T04:56:54.458572+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:56:54.458572+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'6600", 'ondisk_log_start': "45'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13070336, 'num_objects': 3191, 'num_object_clones': 0, 'num_object_copies': 6382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3191, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13324, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -208}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:58.760 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:56:58.760 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'16785", 'reported_seq': 17023, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:54.313095+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:56:54.313095+0000', 'last_peered': '2024-09-16T04:56:54.313095+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:56:54.313095+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'6700", 'ondisk_log_start': "45'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13500416, 'num_objects': 3296, 'num_object_clones': 0, 'num_object_copies': 6592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13536, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -25}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:58.760 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:56:58.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'16998", 'reported_seq': 17136, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:54.471607+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:56:54.471607+0000', 'last_peered': '2024-09-16T04:56:54.471607+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:56:54.471607+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'6900", 'ondisk_log_start': "45'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13189120, 'num_objects': 3220, 'num_object_clones': 0, 'num_object_copies': 6440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13653, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -245}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:58.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:56:58.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'16630", 'reported_seq': 16733, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:54.263697+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:56:54.263697+0000', 'last_peered': '2024-09-16T04:56:54.263697+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:56:54.263697+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'6600", 'ondisk_log_start': "45'6600", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13296233, 'num_objects': 3247, 'num_object_clones': 0, 'num_object_copies': 6494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3247, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16629, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -160}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:58.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:56:58.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'17220", 'reported_seq': 17333, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:54.297151+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:56:54.297151+0000', 'last_peered': '2024-09-16T04:56:54.297151+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:56:54.297151+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'7200", 'ondisk_log_start': "45'7200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13567803, 'num_objects': 3315, 'num_object_clones': 0, 'num_object_copies': 6630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3315, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17217, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -199}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:56:58.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:56:58.762 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'17096", 'reported_seq': 17246, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:54.586830+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:56:54.586830+0000', 'last_peered': '2024-09-16T04:56:54.586830+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:56:54.586830+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'7000", 'ondisk_log_start': "45'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13599337, 'num_objects': 3321, 'num_object_clones': 0, 'num_object_copies': 6642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3321, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13865, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:56:58.762 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:56:59.713 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:56:59.714 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30500/104100 objects degraded (29.299%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":567,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":52050,"data_bytes":213160202,"bytes_used":1761734656,"bytes_avail":288148557824,"bytes_total":289910292480,"degraded_objects":30500,"degraded_total":104100,"degraded_ratio":0.29298751200768491,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":155,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":390},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:56:59.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:56:59.770 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:00.311 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:00.311 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:00.371 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-16T04:56:58.472779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":213160202,"num_objects":52050,"num_object_clones":0,"num_object_copies":104100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234584,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10243,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160907,"ondisk_log_size":160907,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1720444,"kb_used_data":582776,"kb_used_omap":12,"kb_used_meta":1137587,"kb_avail":281395076,"statfs":{"total":289910292480,"available":288148557824,"internally_reserved":0,"allocated":596762624,"data_stored":595830091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1164889089},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,1,1,8,12,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19202048,"num_objects":-4688,"num_object_clones":0,"num_object_copies":-9376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4688,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004354"},"pg_stats":[{"pgid":"2.f","version":"51'16550","reported_seq":17164,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.580576+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:56:54.580576+0000","last_peered":"2024-09-16T04:56:54.580576+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:54.580576+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6500","ondisk_log_start":"45'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13058048,"num_objects":3188,"num_object_clones":0,"num_object_copies":6376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16550,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":109}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'16827","reported_seq":16953,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.255334+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:54.255334+0000","last_peered":"2024-09-16T04:56:54.255334+0000","last_clean":"2024-09-16T04:56:54.255334+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:54.255334+0000","last_undegraded":"2024-09-16T04:56:54.255334+0000","last_fullsized":"2024-09-16T04:56:54.255334+0000","mapping_epoch":35,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12972032,"num_objects":3167,"num_object_clones":0,"num_object_copies":6334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13796,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'16661","reported_seq":16810,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.275191+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:54.275191+0000","last_peered":"2024-09-16T04:56:54.275191+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:54.275191+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13090840,"num_objects":3197,"num_object_clones":0,"num_object_copies":6394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3197,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13400,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-122}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'16997","reported_seq":17076,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.323317+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:54.323317+0000","last_peered":"2024-09-16T04:56:54.323317+0000","last_clean":"2024-09-16T04:56:54.323317+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:54.323317+0000","last_undegraded":"2024-09-16T04:56:54.323317+0000","last_fullsized":"2024-09-16T04:56:54.323317+0000","mapping_epoch":35,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13709312,"num_objects":3347,"num_object_clones":0,"num_object_copies":6694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16997,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'16910","reported_seq":17139,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.580483+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:54.580483+0000","last_peered":"2024-09-16T04:56:54.580483+0000","last_clean":"2024-09-16T04:56:54.580483+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:54.580483+0000","last_undegraded":"2024-09-16T04:56:54.580483+0000","last_fullsized":"2024-09-16T04:56:54.580483+0000","mapping_epoch":34,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13246464,"num_objects":3234,"num_object_clones":0,"num_object_copies":6468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13942,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17036","reported_seq":17179,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.490168+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:54.490168+0000","last_peered":"2024-09-16T04:56:54.490168+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:54.490168+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13422592,"num_objects":3277,"num_object_clones":0,"num_object_copies":6554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14029,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'16669","reported_seq":16827,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.458572+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:54.458572+0000","last_peered":"2024-09-16T04:56:54.458572+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:54.458572+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13070336,"num_objects":3191,"num_object_clones":0,"num_object_copies":6382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13324,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-208}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'16766","reported_seq":16992,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.559721+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:54.559721+0000","last_peered":"2024-09-16T04:56:54.559721+0000","last_clean":"2024-09-16T04:56:54.559721+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:54.559721+0000","last_undegraded":"2024-09-16T04:56:54.559721+0000","last_fullsized":"2024-09-16T04:56:54.559721+0000","mapping_epoch":34,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13312000,"num_objects":3250,"num_object_clones":0,"num_object_copies":6500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13836,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'16785","reported_seq":17023,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.313095+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:54.313095+0000","last_peered":"2024-09-16T04:56:54.313095+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:54.313095+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":3296,"num_object_clones":0,"num_object_copies":6592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13536,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-25}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715089+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715089+0000","last_peered":"2024-09-16T04:56:42.715089+0000","last_clean":"2024-09-16T04:56:42.715089+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715089+0000","last_undegraded":"2024-09-16T04:56:42.715089+0000","last_fullsized":"2024-09-16T04:56:42.715089+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'16998","reported_seq":17136,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.471607+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:54.471607+0000","last_peered":"2024-09-16T04:56:54.471607+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:54.471607+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13189120,"num_objects":3220,"num_object_clones":0,"num_object_copies":6440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13653,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-245}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'16688","reported_seq":23543,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.180425+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:56:54.180425+0000","last_peered":"2024-09-16T04:56:54.180425+0000","last_clean":"2024-09-16T04:56:54.180425+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:56:54.180425+0000","last_undegraded":"2024-09-16T04:56:54.180425+0000","last_fullsized":"2024-09-16T04:56:54.180425+0000","mapping_epoch":50,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:59.009028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13481170,"num_objects":3293,"num_object_clones":0,"num_object_copies":6586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16660,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'16770","reported_seq":23637,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.269131+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:54.269131+0000","last_peered":"2024-09-16T04:56:54.269131+0000","last_clean":"2024-09-16T04:56:54.269131+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:54.269131+0000","last_undegraded":"2024-09-16T04:56:54.269131+0000","last_fullsized":"2024-09-16T04:56:54.269131+0000","mapping_epoch":46,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13139968,"num_objects":3208,"num_object_clones":0,"num_object_copies":6416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'16630","reported_seq":16733,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.263697+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:54.263697+0000","last_peered":"2024-09-16T04:56:54.263697+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:54.263697+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6600","ondisk_log_start":"45'6600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13296233,"num_objects":3247,"num_object_clones":0,"num_object_copies":6494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16629,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-160}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17220","reported_seq":17333,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.297151+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:54.297151+0000","last_peered":"2024-09-16T04:56:54.297151+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:54.297151+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13567803,"num_objects":3315,"num_object_clones":0,"num_object_copies":6630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17217,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-199}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715406+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715406+0000","last_peered":"2024-09-16T04:56:42.715406+0000","last_clean":"2024-09-16T04:56:42.715406+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715406+0000","last_undegraded":"2024-09-16T04:56:42.715406+0000","last_fullsized":"2024-09-16T04:56:42.715406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'16901","reported_seq":23704,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:54.003235+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:54.003235+0000","last_peered":"2024-09-16T04:56:54.003235+0000","last_clean":"2024-09-16T04:56:54.003235+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:54.003235+0000","last_undegraded":"2024-09-16T04:56:54.003235+0000","last_fullsized":"2024-09-16T04:56:54.003235+0000","mapping_epoch":48,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13504512,"num_objects":3297,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13655,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17096","reported_seq":17246,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:54.586830+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:54.586830+0000","last_peered":"2024-09-16T04:56:54.586830+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:54.586830+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13599337,"num_objects":3321,"num_object_clones":0,"num_object_copies":6642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13865,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":213160183,"num_objects":52048,"num_object_clones":0,"num_object_copies":104096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52048,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234582,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10242,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":593588224,"data_stored":593513979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587704,"kb_used_data":213092,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93784136,"statfs":{"total":96636764160,"available":96034955264,"internally_reserved":0,"allocated":218206208,"data_stored":217912561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":383577769},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593816,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551232,"kb_used_data":158564,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93820608,"statfs":{"total":96636764160,"available":96072302592,"internally_reserved":0,"allocated":162369536,"data_stored":162020225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":402058926},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,5,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581508,"kb_used_data":211120,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93790332,"statfs":{"total":96636764160,"available":96041299968,"internally_reserved":0,"allocated":216186880,"data_stored":215897305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":215629824,"data_stored":215626345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":160591872,"data_stored":160528657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217366528,"data_stored":217358977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:03.373 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:04.173 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:04.239 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-16T04:57:02.473699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":203481354,"num_objects":49687,"num_object_clones":0,"num_object_copies":99374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30414,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49687,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236947,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10329,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160670,"ondisk_log_size":160670,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1736104,"kb_used_data":563756,"kb_used_omap":12,"kb_used_meta":1172275,"kb_avail":281379416,"statfs":{"total":289910292480,"available":288132521984,"internally_reserved":0,"allocated":577286144,"data_stored":576353611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1200409601},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,4,5,6,12,2,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19234816,"num_objects":-4696,"num_object_clones":0,"num_object_copies":-9392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4696,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004408"},"pg_stats":[{"pgid":"2.f","version":"51'16708","reported_seq":17494,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.606167+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:56:59.606167+0000","last_peered":"2024-09-16T04:56:59.606167+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:59.606167+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12410880,"num_objects":3030,"num_object_clones":0,"num_object_copies":6060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16708,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":37}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'16953","reported_seq":17079,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:59.358604+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:56:59.358604+0000","last_peered":"2024-09-16T04:56:59.358604+0000","last_clean":"2024-09-16T04:56:59.358604+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:56:59.358604+0000","last_undegraded":"2024-09-16T04:56:59.358604+0000","last_fullsized":"2024-09-16T04:56:59.358604+0000","mapping_epoch":35,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12455936,"num_objects":3041,"num_object_clones":0,"num_object_copies":6082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13922,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'16801","reported_seq":16950,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.285686+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:56:59.285686+0000","last_peered":"2024-09-16T04:56:59.285686+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:56:59.285686+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12517400,"num_objects":3057,"num_object_clones":0,"num_object_copies":6114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3057,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13540,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-262}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'17154","reported_seq":17234,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:59.315290+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:56:59.315290+0000","last_peered":"2024-09-16T04:56:59.315290+0000","last_clean":"2024-09-16T04:56:59.315290+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:56:59.315290+0000","last_undegraded":"2024-09-16T04:56:59.315290+0000","last_fullsized":"2024-09-16T04:56:59.315290+0000","mapping_epoch":35,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13066240,"num_objects":3190,"num_object_clones":0,"num_object_copies":6380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17154,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17043","reported_seq":17272,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:59.459684+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:59.459684+0000","last_peered":"2024-09-16T04:56:59.459684+0000","last_clean":"2024-09-16T04:56:59.459684+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:59.459684+0000","last_undegraded":"2024-09-16T04:56:59.459684+0000","last_fullsized":"2024-09-16T04:56:59.459684+0000","mapping_epoch":34,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12701696,"num_objects":3101,"num_object_clones":0,"num_object_copies":6202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14075,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17194","reported_seq":17337,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.600567+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:59.600567+0000","last_peered":"2024-09-16T04:56:59.600567+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:59.600567+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12775424,"num_objects":3119,"num_object_clones":0,"num_object_copies":6238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14187,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-337}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'16818","reported_seq":16972,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.613929+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:59.613929+0000","last_peered":"2024-09-16T04:56:59.613929+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:59.613929+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12460032,"num_objects":3042,"num_object_clones":0,"num_object_copies":6084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13473,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-357}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'16934","reported_seq":17160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:59.513607+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:59.513607+0000","last_peered":"2024-09-16T04:56:59.513607+0000","last_clean":"2024-09-16T04:56:59.513607+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:59.513607+0000","last_undegraded":"2024-09-16T04:56:59.513607+0000","last_fullsized":"2024-09-16T04:56:59.513607+0000","mapping_epoch":34,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12623872,"num_objects":3082,"num_object_clones":0,"num_object_copies":6164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14004,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'16944","reported_seq":17182,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.306176+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:56:59.306176+0000","last_peered":"2024-09-16T04:56:59.306176+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:56:59.306176+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12849152,"num_objects":3137,"num_object_clones":0,"num_object_copies":6274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13695,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-184}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715089+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715089+0000","last_peered":"2024-09-16T04:56:42.715089+0000","last_clean":"2024-09-16T04:56:42.715089+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715089+0000","last_undegraded":"2024-09-16T04:56:42.715089+0000","last_fullsized":"2024-09-16T04:56:42.715089+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17141","reported_seq":17279,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.600185+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:59.600185+0000","last_peered":"2024-09-16T04:56:59.600185+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:59.600185+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12603392,"num_objects":3077,"num_object_clones":0,"num_object_copies":6154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13796,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-388}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'16835","reported_seq":23690,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:59.230520+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:56:59.230520+0000","last_peered":"2024-09-16T04:56:59.230520+0000","last_clean":"2024-09-16T04:56:59.230520+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:56:59.230520+0000","last_undegraded":"2024-09-16T04:56:59.230520+0000","last_fullsized":"2024-09-16T04:56:59.230520+0000","mapping_epoch":50,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:04.027648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12879058,"num_objects":3146,"num_object_clones":0,"num_object_copies":6292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16807,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'16912","reported_seq":23779,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:59.297292+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:56:59.297292+0000","last_peered":"2024-09-16T04:56:59.297292+0000","last_clean":"2024-09-16T04:56:59.297292+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:56:59.297292+0000","last_undegraded":"2024-09-16T04:56:59.297292+0000","last_fullsized":"2024-09-16T04:56:59.297292+0000","mapping_epoch":46,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12558336,"num_objects":3066,"num_object_clones":0,"num_object_copies":6132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13635,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'16761","reported_seq":16869,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.292137+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:56:59.292137+0000","last_peered":"2024-09-16T04:56:59.292137+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:56:59.292137+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12759657,"num_objects":3116,"num_object_clones":0,"num_object_copies":6232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16760,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-291}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17368","reported_seq":17480,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.321438+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:56:59.321438+0000","last_peered":"2024-09-16T04:56:59.321438+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:56:59.321438+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12961595,"num_objects":3167,"num_object_clones":0,"num_object_copies":6334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17365,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-347}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715406+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715406+0000","last_peered":"2024-09-16T04:56:42.715406+0000","last_clean":"2024-09-16T04:56:42.715406+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715406+0000","last_undegraded":"2024-09-16T04:56:42.715406+0000","last_fullsized":"2024-09-16T04:56:42.715406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17045","reported_seq":23849,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:58.846307+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:56:58.846307+0000","last_peered":"2024-09-16T04:56:58.846307+0000","last_clean":"2024-09-16T04:56:58.846307+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:56:58.846307+0000","last_undegraded":"2024-09-16T04:56:58.846307+0000","last_fullsized":"2024-09-16T04:56:58.846307+0000","mapping_epoch":48,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12914688,"num_objects":3153,"num_object_clones":0,"num_object_copies":6306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13799,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17256","reported_seq":17405,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.563578+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:59.563578+0000","last_peered":"2024-09-16T04:56:59.563578+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:59.563578+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12943977,"num_objects":3161,"num_object_clones":0,"num_object_copies":6322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14025,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":203481335,"num_objects":49685,"num_object_clones":0,"num_object_copies":99370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30414,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49685,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236945,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10328,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":574201856,"data_stored":574127611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585976,"kb_used_data":207200,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93785864,"statfs":{"total":96636764160,"available":96036724736,"internally_reserved":0,"allocated":212172800,"data_stored":211879153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":387837609},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,1,3,1,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593817,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570320,"kb_used_data":151292,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93801520,"statfs":{"total":96636764160,"available":96052756480,"internally_reserved":0,"allocated":154923008,"data_stored":154573697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":429059758},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579808,"kb_used_data":205264,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93792032,"statfs":{"total":96636764160,"available":96043040768,"internally_reserved":0,"allocated":210190336,"data_stored":209900761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":209551360,"data_stored":209547881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":153251840,"data_stored":153188625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":211398656,"data_stored":211391105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:04.241 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:05.145 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:05.146 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:05.191 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-16T04:57:04.474110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":197869834,"num_objects":48317,"num_object_clones":0,"num_object_copies":96634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30414,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48317,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":238317,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10329,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160740,"ondisk_log_size":160740,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1718972,"kb_used_data":550764,"kb_used_omap":12,"kb_used_meta":1168115,"kb_avail":281396548,"statfs":{"total":289910292480,"available":288150065152,"internally_reserved":0,"allocated":563982336,"data_stored":563049803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1196149761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,7,6,10,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24846336,"num_objects":-6066,"num_object_clones":0,"num_object_copies":-12132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6066,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004302"},"pg_stats":[{"pgid":"2.f","version":"51'16708","reported_seq":17494,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.606167+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:56:59.606167+0000","last_peered":"2024-09-16T04:56:59.606167+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:56:59.606167+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6700","ondisk_log_start":"45'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12410880,"num_objects":3030,"num_object_clones":0,"num_object_copies":6060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16708,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":37}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17099","reported_seq":17225,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:04.330035+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:04.330035+0000","last_peered":"2024-09-16T04:57:04.330035+0000","last_clean":"2024-09-16T04:57:04.330035+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:04.330035+0000","last_undegraded":"2024-09-16T04:57:04.330035+0000","last_fullsized":"2024-09-16T04:57:04.330035+0000","mapping_epoch":35,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11857920,"num_objects":2895,"num_object_clones":0,"num_object_copies":5790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14068,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'16964","reported_seq":17113,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:04.288216+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:04.288216+0000","last_peered":"2024-09-16T04:57:04.288216+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:04.288216+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11849752,"num_objects":2894,"num_object_clones":0,"num_object_copies":5788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2894,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13703,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-425}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'17304","reported_seq":17384,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:04.304203+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:04.304203+0000","last_peered":"2024-09-16T04:57:04.304203+0000","last_clean":"2024-09-16T04:57:04.304203+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:04.304203+0000","last_undegraded":"2024-09-16T04:57:04.304203+0000","last_fullsized":"2024-09-16T04:57:04.304203+0000","mapping_epoch":35,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12451840,"num_objects":3040,"num_object_clones":0,"num_object_copies":6080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17304,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17043","reported_seq":17272,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:59.459684+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:56:59.459684+0000","last_peered":"2024-09-16T04:56:59.459684+0000","last_clean":"2024-09-16T04:56:59.459684+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:56:59.459684+0000","last_undegraded":"2024-09-16T04:56:59.459684+0000","last_fullsized":"2024-09-16T04:56:59.459684+0000","mapping_epoch":34,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12701696,"num_objects":3101,"num_object_clones":0,"num_object_copies":6202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14075,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17194","reported_seq":17337,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.600567+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:56:59.600567+0000","last_peered":"2024-09-16T04:56:59.600567+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:56:59.600567+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12775424,"num_objects":3119,"num_object_clones":0,"num_object_copies":6238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14187,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-337}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'16818","reported_seq":16972,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.613929+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:56:59.613929+0000","last_peered":"2024-09-16T04:56:59.613929+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:56:59.613929+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12460032,"num_objects":3042,"num_object_clones":0,"num_object_copies":6084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13473,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-357}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'16934","reported_seq":17160,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:59.513607+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:56:59.513607+0000","last_peered":"2024-09-16T04:56:59.513607+0000","last_clean":"2024-09-16T04:56:59.513607+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:56:59.513607+0000","last_undegraded":"2024-09-16T04:56:59.513607+0000","last_fullsized":"2024-09-16T04:56:59.513607+0000","mapping_epoch":34,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12623872,"num_objects":3082,"num_object_clones":0,"num_object_copies":6164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14004,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17100","reported_seq":17338,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:04.343851+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:04.343851+0000","last_peered":"2024-09-16T04:57:04.343851+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:04.343851+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12210176,"num_objects":2981,"num_object_clones":0,"num_object_copies":5962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13851,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-340}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17141","reported_seq":17279,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.600185+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:56:59.600185+0000","last_peered":"2024-09-16T04:56:59.600185+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:56:59.600185+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12603392,"num_objects":3077,"num_object_clones":0,"num_object_copies":6154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13796,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-388}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'16990","reported_seq":23843,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:04.288172+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:04.288172+0000","last_peered":"2024-09-16T04:57:04.288172+0000","last_clean":"2024-09-16T04:57:04.288172+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:04.288172+0000","last_undegraded":"2024-09-16T04:57:04.288172+0000","last_fullsized":"2024-09-16T04:57:04.288172+0000","mapping_epoch":50,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12244178,"num_objects":2991,"num_object_clones":0,"num_object_copies":5982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16962,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17049","reported_seq":23916,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:04.348626+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:04.348626+0000","last_peered":"2024-09-16T04:57:04.348626+0000","last_clean":"2024-09-16T04:57:04.348626+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:04.348626+0000","last_undegraded":"2024-09-16T04:57:04.348626+0000","last_fullsized":"2024-09-16T04:57:04.348626+0000","mapping_epoch":46,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11997184,"num_objects":2929,"num_object_clones":0,"num_object_copies":5858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13772,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'16920","reported_seq":17028,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:04.320569+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:04.320569+0000","last_peered":"2024-09-16T04:57:04.320569+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:04.320569+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12108393,"num_objects":2957,"num_object_clones":0,"num_object_copies":5914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16919,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-450}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17510","reported_seq":17623,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:04.269880+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:04.269880+0000","last_peered":"2024-09-16T04:57:04.269880+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:04.269880+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12379963,"num_objects":3025,"num_object_clones":0,"num_object_copies":6050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17507,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-489}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715406+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715406+0000","last_peered":"2024-09-16T04:56:42.715406+0000","last_clean":"2024-09-16T04:56:42.715406+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715406+0000","last_undegraded":"2024-09-16T04:56:42.715406+0000","last_fullsized":"2024-09-16T04:56:42.715406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17207","reported_seq":24011,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:03.981996+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:03.981996+0000","last_peered":"2024-09-16T04:57:03.981996+0000","last_clean":"2024-09-16T04:57:03.981996+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:03.981996+0000","last_undegraded":"2024-09-16T04:57:03.981996+0000","last_fullsized":"2024-09-16T04:57:03.981996+0000","mapping_epoch":48,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12251136,"num_objects":2991,"num_object_clones":0,"num_object_copies":5982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13961,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17256","reported_seq":17405,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:56:59.563578+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:56:59.563578+0000","last_peered":"2024-09-16T04:56:59.563578+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:56:59.563578+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12943977,"num_objects":3161,"num_object_clones":0,"num_object_copies":6322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14025,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":197869815,"num_objects":48315,"num_object_clones":0,"num_object_copies":96630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30414,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48315,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":238315,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10328,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":560693248,"data_stored":560619003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584224,"kb_used_data":201296,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93787616,"statfs":{"total":96636764160,"available":96038518784,"internally_reserved":0,"allocated":206127104,"data_stored":205833457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":392097449},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593818,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554940,"kb_used_data":144204,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93816900,"statfs":{"total":96636764160,"available":96068505600,"internally_reserved":0,"allocated":147664896,"data_stored":147315585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":420540078},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579808,"kb_used_data":205264,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93792032,"statfs":{"total":96636764160,"available":96043040768,"internally_reserved":0,"allocated":210190336,"data_stored":209900761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":209551360,"data_stored":209547881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":145891328,"data_stored":145828113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":205250560,"data_stored":205243009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:05.192 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:57:05.193 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'16708", 'reported_seq': 17494, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:59.606167+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:56:59.606167+0000', 'last_peered': '2024-09-16T04:56:59.606167+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:56:59.606167+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'6700", 'ondisk_log_start': "45'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12410880, 'num_objects': 3030, 'num_object_clones': 0, 'num_object_copies': 6060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2993, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3030, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16708, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 349, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 37}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:05.193 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:57:05.193 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'16964", 'reported_seq': 17113, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:04.288216+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:57:04.288216+0000', 'last_peered': '2024-09-16T04:57:04.288216+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:57:04.288216+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'6900", 'ondisk_log_start': "45'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11849752, 'num_objects': 2894, 'num_object_clones': 0, 'num_object_copies': 5788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2894, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13703, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -425}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:05.193 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:57:05.193 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'17194", 'reported_seq': 17337, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:59.600567+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:56:59.600567+0000', 'last_peered': '2024-09-16T04:56:59.600567+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:56:59.600567+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'7100", 'ondisk_log_start': "45'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12775424, 'num_objects': 3119, 'num_object_clones': 0, 'num_object_copies': 6238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3119, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14187, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -337}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:05.193 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:57:05.193 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'16818", 'reported_seq': 16972, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:59.613929+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:56:59.613929+0000', 'last_peered': '2024-09-16T04:56:59.613929+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:56:59.613929+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'6800", 'ondisk_log_start': "45'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12460032, 'num_objects': 3042, 'num_object_clones': 0, 'num_object_copies': 6084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3042, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13473, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -357}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:05.193 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:57:05.193 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'17100", 'reported_seq': 17338, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:04.343851+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:57:04.343851+0000', 'last_peered': '2024-09-16T04:57:04.343851+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:57:04.343851+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'7000", 'ondisk_log_start': "45'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12210176, 'num_objects': 2981, 'num_object_clones': 0, 'num_object_copies': 5962, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2981, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13851, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -340}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:05.194 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:57:05.194 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'17141", 'reported_seq': 17279, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:59.600185+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:56:59.600185+0000', 'last_peered': '2024-09-16T04:56:59.600185+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:56:59.600185+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'7100", 'ondisk_log_start': "45'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12603392, 'num_objects': 3077, 'num_object_clones': 0, 'num_object_copies': 6154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3077, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13796, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -388}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:05.194 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:57:05.194 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'16920", 'reported_seq': 17028, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:04.320569+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:57:04.320569+0000', 'last_peered': '2024-09-16T04:57:04.320569+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:57:04.320569+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'6900", 'ondisk_log_start': "45'6900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12108393, 'num_objects': 2957, 'num_object_clones': 0, 'num_object_copies': 5914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2957, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16919, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -450}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:05.194 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:57:05.194 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'17510", 'reported_seq': 17623, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:04.269880+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:57:04.269880+0000', 'last_peered': '2024-09-16T04:57:04.269880+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:57:04.269880+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'7500", 'ondisk_log_start': "45'7500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12379963, 'num_objects': 3025, 'num_object_clones': 0, 'num_object_copies': 6050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3025, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17507, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -489}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:05.194 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:57:05.194 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'17256", 'reported_seq': 17405, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:56:59.563578+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:56:59.563578+0000', 'last_peered': '2024-09-16T04:56:59.563578+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:56:59.563578+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'7200", 'ondisk_log_start': "45'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12943977, 'num_objects': 3161, 'num_object_clones': 0, 'num_object_copies': 6322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3161, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14025, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:05.194 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:57:06.276 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:06.276 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30414/96634 objects degraded (31.473%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":574,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":48317,"data_bytes":197869834,"bytes_used":1760227328,"bytes_avail":288150065152,"bytes_total":289910292480,"degraded_objects":30414,"degraded_total":96634,"degraded_ratio":0.31473394457437343,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":505},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:57:06.312 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:57:06.312 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:07.171 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:07.172 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:07.217 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-16T04:57:06.474603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":193745162,"num_objects":47310,"num_object_clones":0,"num_object_copies":94620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47310,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239324,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10409,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1721488,"kb_used_data":544960,"kb_used_omap":12,"kb_used_meta":1176435,"kb_avail":281394032,"statfs":{"total":289910292480,"available":288147488768,"internally_reserved":0,"allocated":558039040,"data_stored":557106507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1204669441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,10,7,13,13,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23498752,"num_objects":-5737,"num_object_clones":0,"num_object_copies":-11474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-251,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5737,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004360"},"pg_stats":[{"pgid":"2.f","version":"51'16860","reported_seq":17804,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:04.638674+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:04.638674+0000","last_peered":"2024-09-16T04:57:04.638674+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:04.638674+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'6800","ondisk_log_start":"45'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11788288,"num_objects":2878,"num_object_clones":0,"num_object_copies":5756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16860,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-35}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17099","reported_seq":17225,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:04.330035+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:04.330035+0000","last_peered":"2024-09-16T04:57:04.330035+0000","last_clean":"2024-09-16T04:57:04.330035+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:04.330035+0000","last_undegraded":"2024-09-16T04:57:04.330035+0000","last_fullsized":"2024-09-16T04:57:04.330035+0000","mapping_epoch":35,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11857920,"num_objects":2895,"num_object_clones":0,"num_object_copies":5790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14068,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'16964","reported_seq":17113,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:04.288216+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:04.288216+0000","last_peered":"2024-09-16T04:57:04.288216+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:04.288216+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11849752,"num_objects":2894,"num_object_clones":0,"num_object_copies":5788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2894,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13703,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-425}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'17304","reported_seq":17384,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:04.304203+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:04.304203+0000","last_peered":"2024-09-16T04:57:04.304203+0000","last_clean":"2024-09-16T04:57:04.304203+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:04.304203+0000","last_undegraded":"2024-09-16T04:57:04.304203+0000","last_fullsized":"2024-09-16T04:57:04.304203+0000","mapping_epoch":35,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12451840,"num_objects":3040,"num_object_clones":0,"num_object_copies":6080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17304,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17179","reported_seq":17408,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:04.465055+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:04.465055+0000","last_peered":"2024-09-16T04:57:04.465055+0000","last_clean":"2024-09-16T04:57:04.465055+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:04.465055+0000","last_undegraded":"2024-09-16T04:57:04.465055+0000","last_fullsized":"2024-09-16T04:57:04.465055+0000","mapping_epoch":34,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12144640,"num_objects":2965,"num_object_clones":0,"num_object_copies":5930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14211,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17322","reported_seq":17464,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:04.618607+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:04.618607+0000","last_peered":"2024-09-16T04:57:04.618607+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:04.618607+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12251136,"num_objects":2991,"num_object_clones":0,"num_object_copies":5982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14315,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'16967","reported_seq":17125,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:04.575565+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:04.575565+0000","last_peered":"2024-09-16T04:57:04.575565+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:04.575565+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11849728,"num_objects":2893,"num_object_clones":0,"num_object_copies":5786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13622,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-506}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17089","reported_seq":17315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:04.612809+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:04.612809+0000","last_peered":"2024-09-16T04:57:04.612809+0000","last_clean":"2024-09-16T04:57:04.612809+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:04.612809+0000","last_undegraded":"2024-09-16T04:57:04.612809+0000","last_fullsized":"2024-09-16T04:57:04.612809+0000","mapping_epoch":34,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11988992,"num_objects":2927,"num_object_clones":0,"num_object_copies":5854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14159,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17100","reported_seq":17338,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:04.343851+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:04.343851+0000","last_peered":"2024-09-16T04:57:04.343851+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:04.343851+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12210176,"num_objects":2981,"num_object_clones":0,"num_object_copies":5962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13851,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-340}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17279","reported_seq":17419,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:04.617500+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:04.617500+0000","last_peered":"2024-09-16T04:57:04.617500+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:04.617500+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12038144,"num_objects":2939,"num_object_clones":0,"num_object_copies":5878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13934,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-526}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'16990","reported_seq":23843,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:04.288172+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:04.288172+0000","last_peered":"2024-09-16T04:57:04.288172+0000","last_clean":"2024-09-16T04:57:04.288172+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:04.288172+0000","last_undegraded":"2024-09-16T04:57:04.288172+0000","last_fullsized":"2024-09-16T04:57:04.288172+0000","mapping_epoch":50,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12244178,"num_objects":2991,"num_object_clones":0,"num_object_copies":5982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16962,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17049","reported_seq":23916,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:04.348626+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:04.348626+0000","last_peered":"2024-09-16T04:57:04.348626+0000","last_clean":"2024-09-16T04:57:04.348626+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:04.348626+0000","last_undegraded":"2024-09-16T04:57:04.348626+0000","last_fullsized":"2024-09-16T04:57:04.348626+0000","mapping_epoch":46,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11997184,"num_objects":2929,"num_object_clones":0,"num_object_copies":5858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13772,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'16920","reported_seq":17028,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:04.320569+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:04.320569+0000","last_peered":"2024-09-16T04:57:04.320569+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:04.320569+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'6900","ondisk_log_start":"45'6900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12108393,"num_objects":2957,"num_object_clones":0,"num_object_copies":5914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16919,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-450}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17510","reported_seq":17623,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:04.269880+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:04.269880+0000","last_peered":"2024-09-16T04:57:04.269880+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:04.269880+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12379963,"num_objects":3025,"num_object_clones":0,"num_object_copies":6050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17507,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-489}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715406+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715406+0000","last_peered":"2024-09-16T04:56:42.715406+0000","last_clean":"2024-09-16T04:56:42.715406+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715406+0000","last_undegraded":"2024-09-16T04:56:42.715406+0000","last_fullsized":"2024-09-16T04:56:42.715406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17207","reported_seq":24011,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:03.981996+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:03.981996+0000","last_peered":"2024-09-16T04:57:03.981996+0000","last_clean":"2024-09-16T04:57:03.981996+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:03.981996+0000","last_undegraded":"2024-09-16T04:57:03.981996+0000","last_fullsized":"2024-09-16T04:57:03.981996+0000","mapping_epoch":48,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12251136,"num_objects":2991,"num_object_clones":0,"num_object_copies":5982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13961,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17405","reported_seq":17555,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:04.533546+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:04.533546+0000","last_peered":"2024-09-16T04:57:04.533546+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:04.533546+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12333673,"num_objects":3012,"num_object_clones":0,"num_object_copies":6024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14174,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":193745143,"num_objects":47308,"num_object_clones":0,"num_object_copies":94616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30334,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239322,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10408,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":554692608,"data_stored":554618363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160844,"ondisk_log_size":160844,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584224,"kb_used_data":201296,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93787616,"statfs":{"total":96636764160,"available":96038518784,"internally_reserved":0,"allocated":206127104,"data_stored":205833457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":392097449},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593818,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554940,"kb_used_data":144204,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93816900,"statfs":{"total":96636764160,"available":96068505600,"internally_reserved":0,"allocated":147664896,"data_stored":147315585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":420540078},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582324,"kb_used_data":199460,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93789516,"statfs":{"total":96636764160,"available":96040464384,"internally_reserved":0,"allocated":204247040,"data_stored":203957465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,6,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":203550720,"data_stored":203547241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":145891328,"data_stored":145828113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":205250560,"data_stored":205243009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:10.220 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:11.122 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:11.124 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:11.185 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-16T04:57:10.477402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":184078602,"num_objects":44950,"num_object_clones":0,"num_object_copies":89900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44950,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241684,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10486,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1704260,"kb_used_data":526356,"kb_used_omap":12,"kb_used_meta":1177843,"kb_avail":281411260,"statfs":{"total":289910292480,"available":288165130240,"internally_reserved":0,"allocated":538988544,"data_stored":538056011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1206111233},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,4,3,6,5,9,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29081600,"num_objects":-7100,"num_object_clones":0,"num_object_copies":-14200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7100,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004623"},"pg_stats":[{"pgid":"2.f","version":"51'17017","reported_seq":18116,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.694517+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:09.694517+0000","last_peered":"2024-09-16T04:57:09.694517+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:09.694517+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11145216,"num_objects":2721,"num_object_clones":0,"num_object_copies":5442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17017,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":506,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-115}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17233","reported_seq":17359,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.348890+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:09.348890+0000","last_peered":"2024-09-16T04:57:09.348890+0000","last_clean":"2024-09-16T04:57:09.348890+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:09.348890+0000","last_undegraded":"2024-09-16T04:57:09.348890+0000","last_fullsized":"2024-09-16T04:57:09.348890+0000","mapping_epoch":35,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11309056,"num_objects":2761,"num_object_clones":0,"num_object_copies":5522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14202,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17106","reported_seq":17255,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.410073+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:09.410073+0000","last_peered":"2024-09-16T04:57:09.410073+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:09.410073+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268120,"num_objects":2752,"num_object_clones":0,"num_object_copies":5504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13845,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'17447","reported_seq":17527,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.342241+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:09.342241+0000","last_peered":"2024-09-16T04:57:09.342241+0000","last_clean":"2024-09-16T04:57:09.342241+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:09.342241+0000","last_undegraded":"2024-09-16T04:57:09.342241+0000","last_fullsized":"2024-09-16T04:57:09.342241+0000","mapping_epoch":35,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11866112,"num_objects":2897,"num_object_clones":0,"num_object_copies":5794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17447,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17340","reported_seq":17569,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.646920+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:09.646920+0000","last_peered":"2024-09-16T04:57:09.646920+0000","last_clean":"2024-09-16T04:57:09.646920+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:09.646920+0000","last_undegraded":"2024-09-16T04:57:09.646920+0000","last_fullsized":"2024-09-16T04:57:09.646920+0000","mapping_epoch":34,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11485184,"num_objects":2804,"num_object_clones":0,"num_object_copies":5608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14372,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17451","reported_seq":17594,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.663165+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:09.663165+0000","last_peered":"2024-09-16T04:57:09.663165+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:09.663165+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11722752,"num_objects":2862,"num_object_clones":0,"num_object_copies":5724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14444,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-594}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17098","reported_seq":17256,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.669588+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:09.669588+0000","last_peered":"2024-09-16T04:57:09.669588+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:09.669588+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11313152,"num_objects":2762,"num_object_clones":0,"num_object_copies":5524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13753,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-637}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17235","reported_seq":17460,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.573011+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:09.573011+0000","last_peered":"2024-09-16T04:57:09.573011+0000","last_clean":"2024-09-16T04:57:09.573011+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:09.573011+0000","last_undegraded":"2024-09-16T04:57:09.573011+0000","last_fullsized":"2024-09-16T04:57:09.573011+0000","mapping_epoch":34,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11390976,"num_objects":2781,"num_object_clones":0,"num_object_copies":5562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14305,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17254","reported_seq":17492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.354824+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:09.354824+0000","last_peered":"2024-09-16T04:57:09.354824+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:09.354824+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11579392,"num_objects":2827,"num_object_clones":0,"num_object_copies":5654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14005,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-494}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17426","reported_seq":17566,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.667662+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:09.667662+0000","last_peered":"2024-09-16T04:57:09.667662+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:09.667662+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11436032,"num_objects":2792,"num_object_clones":0,"num_object_copies":5584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14081,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-673}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17134","reported_seq":23989,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.367128+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:09.367128+0000","last_peered":"2024-09-16T04:57:09.367128+0000","last_clean":"2024-09-16T04:57:09.367128+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:09.367128+0000","last_undegraded":"2024-09-16T04:57:09.367128+0000","last_fullsized":"2024-09-16T04:57:09.367128+0000","mapping_epoch":50,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:09.977414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11654354,"num_objects":2847,"num_object_clones":0,"num_object_copies":5694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17106,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17185","reported_seq":24052,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.371359+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:09.371359+0000","last_peered":"2024-09-16T04:57:09.371359+0000","last_clean":"2024-09-16T04:57:09.371359+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:09.371359+0000","last_undegraded":"2024-09-16T04:57:09.371359+0000","last_fullsized":"2024-09-16T04:57:09.371359+0000","mapping_epoch":46,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11440128,"num_objects":2793,"num_object_clones":0,"num_object_copies":5586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13908,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17074","reported_seq":17181,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.406126+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:09.406126+0000","last_peered":"2024-09-16T04:57:09.406126+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:09.406126+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11477609,"num_objects":2803,"num_object_clones":0,"num_object_copies":5606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17073,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-604}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17665","reported_seq":17777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.400977+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:09.400977+0000","last_peered":"2024-09-16T04:57:09.400977+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:09.400977+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'7600","ondisk_log_start":"45'7600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11745083,"num_objects":2870,"num_object_clones":0,"num_object_copies":5740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17662,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-644}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17363","reported_seq":24166,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.009827+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:09.009827+0000","last_peered":"2024-09-16T04:57:09.009827+0000","last_clean":"2024-09-16T04:57:09.009827+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:09.009827+0000","last_undegraded":"2024-09-16T04:57:09.009827+0000","last_fullsized":"2024-09-16T04:57:09.009827+0000","mapping_epoch":48,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11612160,"num_objects":2835,"num_object_clones":0,"num_object_copies":5670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14117,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17576","reported_seq":17726,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.671423+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:09.671423+0000","last_peered":"2024-09-16T04:57:09.671423+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:09.671423+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11633257,"num_objects":2841,"num_object_clones":0,"num_object_copies":5682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14345,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":184078583,"num_objects":44948,"num_object_clones":0,"num_object_copies":89896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44948,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241682,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10485,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":535408640,"data_stored":535334395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570032,"kb_used_data":195424,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93801808,"statfs":{"total":96636764160,"available":96053051392,"internally_reserved":0,"allocated":200114176,"data_stored":199820529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":383577769},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,5,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593819,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570204,"kb_used_data":137292,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93801636,"statfs":{"total":96636764160,"available":96052875264,"internally_reserved":0,"allocated":140587008,"data_stored":140237697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443281070},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,3,0,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564024,"kb_used_data":193640,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93807816,"statfs":{"total":96636764160,"available":96059203584,"internally_reserved":0,"allocated":198287360,"data_stored":197997785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":197488640,"data_stored":197485161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":138690560,"data_stored":138627345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":199229440,"data_stored":199221889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:11.187 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:11.894 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:11.933 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-16T04:57:10.477402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":184078602,"num_objects":44950,"num_object_clones":0,"num_object_copies":89900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44950,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241684,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10486,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1704260,"kb_used_data":526356,"kb_used_omap":12,"kb_used_meta":1177843,"kb_avail":281411260,"statfs":{"total":289910292480,"available":288165130240,"internally_reserved":0,"allocated":538988544,"data_stored":538056011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1206111233},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,4,3,6,5,9,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29081600,"num_objects":-7100,"num_object_clones":0,"num_object_copies":-14200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7100,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004623"},"pg_stats":[{"pgid":"2.f","version":"51'17017","reported_seq":18116,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.694517+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:09.694517+0000","last_peered":"2024-09-16T04:57:09.694517+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:09.694517+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11145216,"num_objects":2721,"num_object_clones":0,"num_object_copies":5442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17017,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":506,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-115}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17233","reported_seq":17359,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.348890+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:09.348890+0000","last_peered":"2024-09-16T04:57:09.348890+0000","last_clean":"2024-09-16T04:57:09.348890+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:09.348890+0000","last_undegraded":"2024-09-16T04:57:09.348890+0000","last_fullsized":"2024-09-16T04:57:09.348890+0000","mapping_epoch":35,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11309056,"num_objects":2761,"num_object_clones":0,"num_object_copies":5522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14202,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17106","reported_seq":17255,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.410073+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:09.410073+0000","last_peered":"2024-09-16T04:57:09.410073+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:09.410073+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268120,"num_objects":2752,"num_object_clones":0,"num_object_copies":5504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13845,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'17447","reported_seq":17527,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.342241+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:09.342241+0000","last_peered":"2024-09-16T04:57:09.342241+0000","last_clean":"2024-09-16T04:57:09.342241+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:09.342241+0000","last_undegraded":"2024-09-16T04:57:09.342241+0000","last_fullsized":"2024-09-16T04:57:09.342241+0000","mapping_epoch":35,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11866112,"num_objects":2897,"num_object_clones":0,"num_object_copies":5794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17447,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17340","reported_seq":17569,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.646920+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:09.646920+0000","last_peered":"2024-09-16T04:57:09.646920+0000","last_clean":"2024-09-16T04:57:09.646920+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:09.646920+0000","last_undegraded":"2024-09-16T04:57:09.646920+0000","last_fullsized":"2024-09-16T04:57:09.646920+0000","mapping_epoch":34,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11485184,"num_objects":2804,"num_object_clones":0,"num_object_copies":5608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14372,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17451","reported_seq":17594,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.663165+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:09.663165+0000","last_peered":"2024-09-16T04:57:09.663165+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:09.663165+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11722752,"num_objects":2862,"num_object_clones":0,"num_object_copies":5724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14444,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-594}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17098","reported_seq":17256,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.669588+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:09.669588+0000","last_peered":"2024-09-16T04:57:09.669588+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:09.669588+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11313152,"num_objects":2762,"num_object_clones":0,"num_object_copies":5524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13753,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-637}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17235","reported_seq":17460,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.573011+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:09.573011+0000","last_peered":"2024-09-16T04:57:09.573011+0000","last_clean":"2024-09-16T04:57:09.573011+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:09.573011+0000","last_undegraded":"2024-09-16T04:57:09.573011+0000","last_fullsized":"2024-09-16T04:57:09.573011+0000","mapping_epoch":34,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11390976,"num_objects":2781,"num_object_clones":0,"num_object_copies":5562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14305,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17254","reported_seq":17492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.354824+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:09.354824+0000","last_peered":"2024-09-16T04:57:09.354824+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:09.354824+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11579392,"num_objects":2827,"num_object_clones":0,"num_object_copies":5654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14005,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-494}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17426","reported_seq":17566,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.667662+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:09.667662+0000","last_peered":"2024-09-16T04:57:09.667662+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:09.667662+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11436032,"num_objects":2792,"num_object_clones":0,"num_object_copies":5584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14081,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-673}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17134","reported_seq":23989,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.367128+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:09.367128+0000","last_peered":"2024-09-16T04:57:09.367128+0000","last_clean":"2024-09-16T04:57:09.367128+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:09.367128+0000","last_undegraded":"2024-09-16T04:57:09.367128+0000","last_fullsized":"2024-09-16T04:57:09.367128+0000","mapping_epoch":50,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:09.977414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11654354,"num_objects":2847,"num_object_clones":0,"num_object_copies":5694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17106,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17185","reported_seq":24052,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.371359+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:09.371359+0000","last_peered":"2024-09-16T04:57:09.371359+0000","last_clean":"2024-09-16T04:57:09.371359+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:09.371359+0000","last_undegraded":"2024-09-16T04:57:09.371359+0000","last_fullsized":"2024-09-16T04:57:09.371359+0000","mapping_epoch":46,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11440128,"num_objects":2793,"num_object_clones":0,"num_object_copies":5586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13908,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17074","reported_seq":17181,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.406126+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:09.406126+0000","last_peered":"2024-09-16T04:57:09.406126+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:09.406126+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11477609,"num_objects":2803,"num_object_clones":0,"num_object_copies":5606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17073,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-604}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17665","reported_seq":17777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.400977+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:09.400977+0000","last_peered":"2024-09-16T04:57:09.400977+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:09.400977+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'7600","ondisk_log_start":"45'7600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11745083,"num_objects":2870,"num_object_clones":0,"num_object_copies":5740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17662,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-644}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17363","reported_seq":24166,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.009827+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:09.009827+0000","last_peered":"2024-09-16T04:57:09.009827+0000","last_clean":"2024-09-16T04:57:09.009827+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:09.009827+0000","last_undegraded":"2024-09-16T04:57:09.009827+0000","last_fullsized":"2024-09-16T04:57:09.009827+0000","mapping_epoch":48,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11612160,"num_objects":2835,"num_object_clones":0,"num_object_copies":5670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14117,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17576","reported_seq":17726,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.671423+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:09.671423+0000","last_peered":"2024-09-16T04:57:09.671423+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:09.671423+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11633257,"num_objects":2841,"num_object_clones":0,"num_object_copies":5682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14345,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":184078583,"num_objects":44948,"num_object_clones":0,"num_object_copies":89896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44948,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241682,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10485,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":535408640,"data_stored":535334395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570032,"kb_used_data":195424,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93801808,"statfs":{"total":96636764160,"available":96053051392,"internally_reserved":0,"allocated":200114176,"data_stored":199820529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":383577769},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,5,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593819,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570204,"kb_used_data":137292,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93801636,"statfs":{"total":96636764160,"available":96052875264,"internally_reserved":0,"allocated":140587008,"data_stored":140237697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443281070},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,3,0,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564024,"kb_used_data":193640,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93807816,"statfs":{"total":96636764160,"available":96059203584,"internally_reserved":0,"allocated":198287360,"data_stored":197997785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":197488640,"data_stored":197485161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":138690560,"data_stored":138627345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":199229440,"data_stored":199221889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:11.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:57:11.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'17017", 'reported_seq': 18116, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:09.694517+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:57:09.694517+0000', 'last_peered': '2024-09-16T04:57:09.694517+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:57:09.694517+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'7000", 'ondisk_log_start': "45'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11145216, 'num_objects': 2721, 'num_object_clones': 0, 'num_object_copies': 5442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2836, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17017, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 506, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -115}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:11.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:57:11.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'17106", 'reported_seq': 17255, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:09.410073+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:57:09.410073+0000', 'last_peered': '2024-09-16T04:57:09.410073+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:57:09.410073+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'7100", 'ondisk_log_start': "45'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11268120, 'num_objects': 2752, 'num_object_clones': 0, 'num_object_copies': 5504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2752, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13845, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -567}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:11.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:57:11.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'17451", 'reported_seq': 17594, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:09.663165+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:57:09.663165+0000', 'last_peered': '2024-09-16T04:57:09.663165+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:57:09.663165+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'7400", 'ondisk_log_start': "45'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11722752, 'num_objects': 2862, 'num_object_clones': 0, 'num_object_copies': 5724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2862, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14444, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -594}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:11.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:57:11.935 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'17098", 'reported_seq': 17256, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:09.669588+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:57:09.669588+0000', 'last_peered': '2024-09-16T04:57:09.669588+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:57:09.669588+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'7000", 'ondisk_log_start': "45'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11313152, 'num_objects': 2762, 'num_object_clones': 0, 'num_object_copies': 5524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13753, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -637}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:11.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:57:11.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'17254", 'reported_seq': 17492, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:09.354824+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:57:09.354824+0000', 'last_peered': '2024-09-16T04:57:09.354824+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:57:09.354824+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'7200", 'ondisk_log_start': "45'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11579392, 'num_objects': 2827, 'num_object_clones': 0, 'num_object_copies': 5654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2827, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14005, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -494}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:11.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:57:11.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'17426", 'reported_seq': 17566, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:09.667662+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:57:09.667662+0000', 'last_peered': '2024-09-16T04:57:09.667662+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:57:09.667662+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'7400", 'ondisk_log_start': "45'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11436032, 'num_objects': 2792, 'num_object_clones': 0, 'num_object_copies': 5584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2792, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14081, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -673}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:11.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:57:11.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'17074", 'reported_seq': 17181, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:09.406126+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:57:09.406126+0000', 'last_peered': '2024-09-16T04:57:09.406126+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:57:09.406126+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'7000", 'ondisk_log_start': "45'7000", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11477609, 'num_objects': 2803, 'num_object_clones': 0, 'num_object_copies': 5606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2803, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17073, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -604}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:11.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:57:11.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'17665", 'reported_seq': 17777, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:09.400977+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:57:09.400977+0000', 'last_peered': '2024-09-16T04:57:09.400977+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:57:09.400977+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "45'7600", 'ondisk_log_start': "45'7600", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11745083, 'num_objects': 2870, 'num_object_clones': 0, 'num_object_copies': 5740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2870, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17662, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -644}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:11.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:57:11.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'17576", 'reported_seq': 17726, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:09.671423+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:57:09.671423+0000', 'last_peered': '2024-09-16T04:57:09.671423+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:57:09.671423+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'7500", 'ondisk_log_start': "45'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11633257, 'num_objects': 2841, 'num_object_clones': 0, 'num_object_copies': 5682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2841, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14345, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -683}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:11.937 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:57:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:12.878 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30257/89900 objects degraded (33.656%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":580,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":44950,"data_bytes":184078602,"bytes_used":1745162240,"bytes_avail":288165130240,"bytes_total":289910292480,"degraded_objects":30257,"degraded_total":89900,"degraded_ratio":0.33656284760845379,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":394},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:57:12.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:57:12.957 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:13.854 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:13.858 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:13.930 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-16T04:57:12.477912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":184078602,"num_objects":44950,"num_object_clones":0,"num_object_copies":89900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44950,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241684,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10486,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1704260,"kb_used_data":526356,"kb_used_omap":12,"kb_used_meta":1177843,"kb_avail":281411260,"statfs":{"total":289910292480,"available":288165130240,"internally_reserved":0,"allocated":538988544,"data_stored":538056011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1206111233},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,4,3,6,5,9,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19402752,"num_objects":-4737,"num_object_clones":0,"num_object_copies":-9474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4737,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004591"},"pg_stats":[{"pgid":"2.f","version":"51'17017","reported_seq":18116,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.694517+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:09.694517+0000","last_peered":"2024-09-16T04:57:09.694517+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:09.694517+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11145216,"num_objects":2721,"num_object_clones":0,"num_object_copies":5442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2836,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17017,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":506,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-115}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17233","reported_seq":17359,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.348890+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:09.348890+0000","last_peered":"2024-09-16T04:57:09.348890+0000","last_clean":"2024-09-16T04:57:09.348890+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:09.348890+0000","last_undegraded":"2024-09-16T04:57:09.348890+0000","last_fullsized":"2024-09-16T04:57:09.348890+0000","mapping_epoch":35,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11309056,"num_objects":2761,"num_object_clones":0,"num_object_copies":5522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14202,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17106","reported_seq":17255,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.410073+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:09.410073+0000","last_peered":"2024-09-16T04:57:09.410073+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:09.410073+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268120,"num_objects":2752,"num_object_clones":0,"num_object_copies":5504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13845,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-567}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'17447","reported_seq":17527,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.342241+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:09.342241+0000","last_peered":"2024-09-16T04:57:09.342241+0000","last_clean":"2024-09-16T04:57:09.342241+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:09.342241+0000","last_undegraded":"2024-09-16T04:57:09.342241+0000","last_fullsized":"2024-09-16T04:57:09.342241+0000","mapping_epoch":35,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11866112,"num_objects":2897,"num_object_clones":0,"num_object_copies":5794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17447,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17340","reported_seq":17569,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.646920+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:09.646920+0000","last_peered":"2024-09-16T04:57:09.646920+0000","last_clean":"2024-09-16T04:57:09.646920+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:09.646920+0000","last_undegraded":"2024-09-16T04:57:09.646920+0000","last_fullsized":"2024-09-16T04:57:09.646920+0000","mapping_epoch":34,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11485184,"num_objects":2804,"num_object_clones":0,"num_object_copies":5608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14372,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17451","reported_seq":17594,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.663165+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:09.663165+0000","last_peered":"2024-09-16T04:57:09.663165+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:09.663165+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11722752,"num_objects":2862,"num_object_clones":0,"num_object_copies":5724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14444,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-594}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17098","reported_seq":17256,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.669588+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:09.669588+0000","last_peered":"2024-09-16T04:57:09.669588+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:09.669588+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11313152,"num_objects":2762,"num_object_clones":0,"num_object_copies":5524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13753,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-637}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17235","reported_seq":17460,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.573011+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:09.573011+0000","last_peered":"2024-09-16T04:57:09.573011+0000","last_clean":"2024-09-16T04:57:09.573011+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:09.573011+0000","last_undegraded":"2024-09-16T04:57:09.573011+0000","last_fullsized":"2024-09-16T04:57:09.573011+0000","mapping_epoch":34,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11390976,"num_objects":2781,"num_object_clones":0,"num_object_copies":5562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14305,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17254","reported_seq":17492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.354824+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:09.354824+0000","last_peered":"2024-09-16T04:57:09.354824+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:09.354824+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11579392,"num_objects":2827,"num_object_clones":0,"num_object_copies":5654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14005,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-494}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17426","reported_seq":17566,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.667662+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:09.667662+0000","last_peered":"2024-09-16T04:57:09.667662+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:09.667662+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11436032,"num_objects":2792,"num_object_clones":0,"num_object_copies":5584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14081,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-673}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17134","reported_seq":23989,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.367128+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:09.367128+0000","last_peered":"2024-09-16T04:57:09.367128+0000","last_clean":"2024-09-16T04:57:09.367128+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:09.367128+0000","last_undegraded":"2024-09-16T04:57:09.367128+0000","last_fullsized":"2024-09-16T04:57:09.367128+0000","mapping_epoch":50,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:09.977414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11654354,"num_objects":2847,"num_object_clones":0,"num_object_copies":5694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17106,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17185","reported_seq":24052,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.371359+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:09.371359+0000","last_peered":"2024-09-16T04:57:09.371359+0000","last_clean":"2024-09-16T04:57:09.371359+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:09.371359+0000","last_undegraded":"2024-09-16T04:57:09.371359+0000","last_fullsized":"2024-09-16T04:57:09.371359+0000","mapping_epoch":46,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11440128,"num_objects":2793,"num_object_clones":0,"num_object_copies":5586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13908,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17074","reported_seq":17181,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.406126+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:09.406126+0000","last_peered":"2024-09-16T04:57:09.406126+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:09.406126+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'7000","ondisk_log_start":"45'7000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11477609,"num_objects":2803,"num_object_clones":0,"num_object_copies":5606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17073,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-604}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17665","reported_seq":17777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.400977+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:09.400977+0000","last_peered":"2024-09-16T04:57:09.400977+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:09.400977+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"45'7600","ondisk_log_start":"45'7600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11745083,"num_objects":2870,"num_object_clones":0,"num_object_copies":5740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17662,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-644}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17363","reported_seq":24166,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:09.009827+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:09.009827+0000","last_peered":"2024-09-16T04:57:09.009827+0000","last_clean":"2024-09-16T04:57:09.009827+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:09.009827+0000","last_undegraded":"2024-09-16T04:57:09.009827+0000","last_fullsized":"2024-09-16T04:57:09.009827+0000","mapping_epoch":48,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11612160,"num_objects":2835,"num_object_clones":0,"num_object_copies":5670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14117,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17576","reported_seq":17726,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:09.671423+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:09.671423+0000","last_peered":"2024-09-16T04:57:09.671423+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:09.671423+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11633257,"num_objects":2841,"num_object_clones":0,"num_object_copies":5682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14345,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":184078583,"num_objects":44948,"num_object_clones":0,"num_object_copies":89896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44948,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241682,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10485,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":535408640,"data_stored":535334395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570032,"kb_used_data":195424,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93801808,"statfs":{"total":96636764160,"available":96053051392,"internally_reserved":0,"allocated":200114176,"data_stored":199820529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":383577769},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,5,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593819,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570204,"kb_used_data":137292,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93801636,"statfs":{"total":96636764160,"available":96052875264,"internally_reserved":0,"allocated":140587008,"data_stored":140237697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443281070},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,3,0,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564024,"kb_used_data":193640,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93807816,"statfs":{"total":96636764160,"available":96059203584,"internally_reserved":0,"allocated":198287360,"data_stored":197997785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":379252394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":197488640,"data_stored":197485161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":138690560,"data_stored":138627345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":199229440,"data_stored":199221889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:16.933 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:17.606 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:17.607 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:17.674 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-16T04:57:16.481100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":174588170,"num_objects":42633,"num_object_clones":0,"num_object_copies":85266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244001,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10578,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160724,"ondisk_log_size":160724,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1720452,"kb_used_data":507840,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281395068,"statfs":{"total":289910292480,"available":288148549632,"internally_reserved":0,"allocated":520028160,"data_stored":519095627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1241631745},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,6,6,10,18,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23281664,"num_objects":-5684,"num_object_clones":0,"num_object_copies":-11368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5684,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006989"},"pg_stats":[{"pgid":"2.f","version":"51'17139","reported_seq":18423,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.733284+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:14.733284+0000","last_peered":"2024-09-16T04:57:14.733284+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:14.733284+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10645504,"num_objects":2599,"num_object_clones":0,"num_object_copies":5198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2744,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17139,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":598,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-145}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17351","reported_seq":17477,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.469335+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:14.469335+0000","last_peered":"2024-09-16T04:57:14.469335+0000","last_clean":"2024-09-16T04:57:14.469335+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:14.469335+0000","last_undegraded":"2024-09-16T04:57:14.469335+0000","last_fullsized":"2024-09-16T04:57:14.469335+0000","mapping_epoch":35,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10825728,"num_objects":2643,"num_object_clones":0,"num_object_copies":5286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14320,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17239","reported_seq":17388,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.411104+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:14.411104+0000","last_peered":"2024-09-16T04:57:14.411104+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:14.411104+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10723352,"num_objects":2619,"num_object_clones":0,"num_object_copies":5238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2619,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13978,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-700}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'17596","reported_seq":17676,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.378473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:14.378473+0000","last_peered":"2024-09-16T04:57:14.378473+0000","last_clean":"2024-09-16T04:57:14.378473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:14.378473+0000","last_undegraded":"2024-09-16T04:57:14.378473+0000","last_fullsized":"2024-09-16T04:57:14.378473+0000","mapping_epoch":35,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11255808,"num_objects":2748,"num_object_clones":0,"num_object_copies":5496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17596,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17510","reported_seq":17739,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.637045+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:14.637045+0000","last_peered":"2024-09-16T04:57:14.637045+0000","last_clean":"2024-09-16T04:57:14.637045+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:14.637045+0000","last_undegraded":"2024-09-16T04:57:14.637045+0000","last_fullsized":"2024-09-16T04:57:14.637045+0000","mapping_epoch":34,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10788864,"num_objects":2634,"num_object_clones":0,"num_object_copies":5268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14542,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17605","reported_seq":17749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.725162+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:14.725162+0000","last_peered":"2024-09-16T04:57:14.725162+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:14.725162+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'7600","ondisk_log_start":"45'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11091968,"num_objects":2708,"num_object_clones":0,"num_object_copies":5416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14598,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17244","reported_seq":17403,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.713213+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:14.713213+0000","last_peered":"2024-09-16T04:57:14.713213+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:14.713213+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10715136,"num_objects":2616,"num_object_clones":0,"num_object_copies":5232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13899,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-783}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17375","reported_seq":17601,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.712689+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:14.712689+0000","last_peered":"2024-09-16T04:57:14.712689+0000","last_clean":"2024-09-16T04:57:14.712689+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:14.712689+0000","last_undegraded":"2024-09-16T04:57:14.712689+0000","last_fullsized":"2024-09-16T04:57:14.712689+0000","mapping_epoch":34,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10817536,"num_objects":2641,"num_object_clones":0,"num_object_copies":5282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14445,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17399","reported_seq":17634,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.382628+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:14.382628+0000","last_peered":"2024-09-16T04:57:14.382628+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:14.382628+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10985472,"num_objects":2682,"num_object_clones":0,"num_object_copies":5364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14150,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-639}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17574","reported_seq":17714,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.732779+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:14.732779+0000","last_peered":"2024-09-16T04:57:14.732779+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:14.732779+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10829824,"num_objects":2644,"num_object_clones":0,"num_object_copies":5288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14229,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-821}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17282","reported_seq":24137,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.420091+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:14.420091+0000","last_peered":"2024-09-16T04:57:14.420091+0000","last_clean":"2024-09-16T04:57:14.420091+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:14.420091+0000","last_undegraded":"2024-09-16T04:57:14.420091+0000","last_fullsized":"2024-09-16T04:57:14.420091+0000","mapping_epoch":50,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:15.063964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11048146,"num_objects":2699,"num_object_clones":0,"num_object_copies":5398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17254,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17351","reported_seq":24218,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.445071+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:14.445071+0000","last_peered":"2024-09-16T04:57:14.445071+0000","last_clean":"2024-09-16T04:57:14.445071+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:14.445071+0000","last_undegraded":"2024-09-16T04:57:14.445071+0000","last_fullsized":"2024-09-16T04:57:14.445071+0000","mapping_epoch":46,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10760192,"num_objects":2627,"num_object_clones":0,"num_object_copies":5254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14074,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17206","reported_seq":17314,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.373427+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:14.373427+0000","last_peered":"2024-09-16T04:57:14.373427+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:14.373427+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10936937,"num_objects":2671,"num_object_clones":0,"num_object_copies":5342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17205,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-736}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17816","reported_seq":17929,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.428939+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:14.428939+0000","last_peered":"2024-09-16T04:57:14.428939+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:14.428939+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11126587,"num_objects":2719,"num_object_clones":0,"num_object_copies":5438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-795}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17507","reported_seq":24311,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.092873+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:14.092873+0000","last_peered":"2024-09-16T04:57:14.092873+0000","last_clean":"2024-09-16T04:57:14.092873+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:14.092873+0000","last_undegraded":"2024-09-16T04:57:14.092873+0000","last_fullsized":"2024-09-16T04:57:14.092873+0000","mapping_epoch":48,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11022336,"num_objects":2691,"num_object_clones":0,"num_object_copies":5382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14261,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17727","reported_seq":17877,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.589524+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:14.589524+0000","last_peered":"2024-09-16T04:57:14.589524+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:14.589524+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'7700","ondisk_log_start":"45'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11014761,"num_objects":2690,"num_object_clones":0,"num_object_copies":5380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14496,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":174588151,"num_objects":42631,"num_object_clones":0,"num_object_copies":85262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42631,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":243999,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10577,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516497408,"data_stored":516423163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568572,"kb_used_data":189804,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93803268,"statfs":{"total":96636764160,"available":96054546432,"internally_reserved":0,"allocated":194359296,"data_stored":194065649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":387837609},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,2,3,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593820,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571356,"kb_used_data":130116,"kb_used_omap":4,"kb_used_meta":441211,"kb_avail":93800484,"statfs":{"total":96636764160,"available":96051695616,"internally_reserved":0,"allocated":133238784,"data_stored":132889473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":451800750},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580524,"kb_used_data":187920,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93791316,"statfs":{"total":96636764160,"available":96042307584,"internally_reserved":0,"allocated":192430080,"data_stored":192140505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,2,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":191586304,"data_stored":191582825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131604480,"data_stored":131541265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":193306624,"data_stored":193299073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:17.675 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:18.557 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:18.558 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:18.602 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-16T04:57:18.484786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":174588170,"num_objects":42633,"num_object_clones":0,"num_object_copies":85266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244001,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10578,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160724,"ondisk_log_size":160724,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1720452,"kb_used_data":507840,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281395068,"statfs":{"total":289910292480,"available":288148549632,"internally_reserved":0,"allocated":520028160,"data_stored":519095627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1241631745},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,6,6,10,18,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19156992,"num_objects":-4677,"num_object_clones":0,"num_object_copies":-9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4677,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010183"},"pg_stats":[{"pgid":"2.f","version":"51'17139","reported_seq":18423,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.733284+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:14.733284+0000","last_peered":"2024-09-16T04:57:14.733284+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:14.733284+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10645504,"num_objects":2599,"num_object_clones":0,"num_object_copies":5198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2744,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17139,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":598,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-145}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17351","reported_seq":17477,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.469335+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:14.469335+0000","last_peered":"2024-09-16T04:57:14.469335+0000","last_clean":"2024-09-16T04:57:14.469335+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:14.469335+0000","last_undegraded":"2024-09-16T04:57:14.469335+0000","last_fullsized":"2024-09-16T04:57:14.469335+0000","mapping_epoch":35,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10825728,"num_objects":2643,"num_object_clones":0,"num_object_copies":5286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14320,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17239","reported_seq":17388,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.411104+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:14.411104+0000","last_peered":"2024-09-16T04:57:14.411104+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:14.411104+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10723352,"num_objects":2619,"num_object_clones":0,"num_object_copies":5238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2619,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13978,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-700}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'17596","reported_seq":17676,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.378473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:14.378473+0000","last_peered":"2024-09-16T04:57:14.378473+0000","last_clean":"2024-09-16T04:57:14.378473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:14.378473+0000","last_undegraded":"2024-09-16T04:57:14.378473+0000","last_fullsized":"2024-09-16T04:57:14.378473+0000","mapping_epoch":35,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11255808,"num_objects":2748,"num_object_clones":0,"num_object_copies":5496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17596,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17510","reported_seq":17739,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.637045+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:14.637045+0000","last_peered":"2024-09-16T04:57:14.637045+0000","last_clean":"2024-09-16T04:57:14.637045+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:14.637045+0000","last_undegraded":"2024-09-16T04:57:14.637045+0000","last_fullsized":"2024-09-16T04:57:14.637045+0000","mapping_epoch":34,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10788864,"num_objects":2634,"num_object_clones":0,"num_object_copies":5268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14542,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17605","reported_seq":17749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.725162+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:14.725162+0000","last_peered":"2024-09-16T04:57:14.725162+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:14.725162+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'7600","ondisk_log_start":"45'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11091968,"num_objects":2708,"num_object_clones":0,"num_object_copies":5416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14598,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17244","reported_seq":17403,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.713213+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:14.713213+0000","last_peered":"2024-09-16T04:57:14.713213+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:14.713213+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10715136,"num_objects":2616,"num_object_clones":0,"num_object_copies":5232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13899,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-783}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17375","reported_seq":17601,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.712689+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:14.712689+0000","last_peered":"2024-09-16T04:57:14.712689+0000","last_clean":"2024-09-16T04:57:14.712689+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:14.712689+0000","last_undegraded":"2024-09-16T04:57:14.712689+0000","last_fullsized":"2024-09-16T04:57:14.712689+0000","mapping_epoch":34,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10817536,"num_objects":2641,"num_object_clones":0,"num_object_copies":5282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14445,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17399","reported_seq":17634,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.382628+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:14.382628+0000","last_peered":"2024-09-16T04:57:14.382628+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:14.382628+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10985472,"num_objects":2682,"num_object_clones":0,"num_object_copies":5364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14150,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-639}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17574","reported_seq":17714,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.732779+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:14.732779+0000","last_peered":"2024-09-16T04:57:14.732779+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:14.732779+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10829824,"num_objects":2644,"num_object_clones":0,"num_object_copies":5288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14229,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-821}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17282","reported_seq":24137,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.420091+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:14.420091+0000","last_peered":"2024-09-16T04:57:14.420091+0000","last_clean":"2024-09-16T04:57:14.420091+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:14.420091+0000","last_undegraded":"2024-09-16T04:57:14.420091+0000","last_fullsized":"2024-09-16T04:57:14.420091+0000","mapping_epoch":50,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:15.063964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11048146,"num_objects":2699,"num_object_clones":0,"num_object_copies":5398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17254,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17351","reported_seq":24218,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.445071+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:14.445071+0000","last_peered":"2024-09-16T04:57:14.445071+0000","last_clean":"2024-09-16T04:57:14.445071+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:14.445071+0000","last_undegraded":"2024-09-16T04:57:14.445071+0000","last_fullsized":"2024-09-16T04:57:14.445071+0000","mapping_epoch":46,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10760192,"num_objects":2627,"num_object_clones":0,"num_object_copies":5254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14074,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17206","reported_seq":17314,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.373427+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:14.373427+0000","last_peered":"2024-09-16T04:57:14.373427+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:14.373427+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10936937,"num_objects":2671,"num_object_clones":0,"num_object_copies":5342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17205,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-736}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17816","reported_seq":17929,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.428939+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:14.428939+0000","last_peered":"2024-09-16T04:57:14.428939+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:14.428939+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11126587,"num_objects":2719,"num_object_clones":0,"num_object_copies":5438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-795}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17507","reported_seq":24311,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.092873+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:14.092873+0000","last_peered":"2024-09-16T04:57:14.092873+0000","last_clean":"2024-09-16T04:57:14.092873+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:14.092873+0000","last_undegraded":"2024-09-16T04:57:14.092873+0000","last_fullsized":"2024-09-16T04:57:14.092873+0000","mapping_epoch":48,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11022336,"num_objects":2691,"num_object_clones":0,"num_object_copies":5382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14261,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17727","reported_seq":17877,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.589524+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:14.589524+0000","last_peered":"2024-09-16T04:57:14.589524+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:14.589524+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'7700","ondisk_log_start":"45'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11014761,"num_objects":2690,"num_object_clones":0,"num_object_copies":5380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14496,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":174588151,"num_objects":42631,"num_object_clones":0,"num_object_copies":85262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42631,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":243999,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10577,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516497408,"data_stored":516423163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568572,"kb_used_data":189804,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93803268,"statfs":{"total":96636764160,"available":96054546432,"internally_reserved":0,"allocated":194359296,"data_stored":194065649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":387837609},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,2,3,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593820,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571356,"kb_used_data":130116,"kb_used_omap":4,"kb_used_meta":441211,"kb_avail":93800484,"statfs":{"total":96636764160,"available":96051695616,"internally_reserved":0,"allocated":133238784,"data_stored":132889473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":451800750},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580524,"kb_used_data":187920,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93791316,"statfs":{"total":96636764160,"available":96042307584,"internally_reserved":0,"allocated":192430080,"data_stored":192140505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,2,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":191586304,"data_stored":191582825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131604480,"data_stored":131541265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":193306624,"data_stored":193299073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:18.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:57:18.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'17139", 'reported_seq': 18423, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:14.733284+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:57:14.733284+0000', 'last_peered': '2024-09-16T04:57:14.733284+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:57:14.733284+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'7100", 'ondisk_log_start': "45'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10645504, 'num_objects': 2599, 'num_object_clones': 0, 'num_object_copies': 5198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2744, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2599, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17139, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 598, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -145}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:18.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:57:18.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'17239", 'reported_seq': 17388, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:14.411104+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:57:14.411104+0000', 'last_peered': '2024-09-16T04:57:14.411104+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:57:14.411104+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'7200", 'ondisk_log_start': "45'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10723352, 'num_objects': 2619, 'num_object_clones': 0, 'num_object_copies': 5238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2619, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13978, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -700}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:18.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:57:18.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'17605", 'reported_seq': 17749, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:14.725162+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:57:14.725162+0000', 'last_peered': '2024-09-16T04:57:14.725162+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:57:14.725162+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "45'7600", 'ondisk_log_start': "45'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11091968, 'num_objects': 2708, 'num_object_clones': 0, 'num_object_copies': 5416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2708, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14598, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -748}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:18.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:57:18.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'17244", 'reported_seq': 17403, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:14.713213+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:57:14.713213+0000', 'last_peered': '2024-09-16T04:57:14.713213+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:57:14.713213+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'7200", 'ondisk_log_start': "45'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10715136, 'num_objects': 2616, 'num_object_clones': 0, 'num_object_copies': 5232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2616, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13899, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -783}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:18.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:57:18.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'17399", 'reported_seq': 17634, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:14.382628+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:57:14.382628+0000', 'last_peered': '2024-09-16T04:57:14.382628+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:57:14.382628+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'7300", 'ondisk_log_start': "45'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10985472, 'num_objects': 2682, 'num_object_clones': 0, 'num_object_copies': 5364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2682, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14150, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -639}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:18.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:57:18.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'17574", 'reported_seq': 17714, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:14.732779+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:57:14.732779+0000', 'last_peered': '2024-09-16T04:57:14.732779+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:57:14.732779+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'7500", 'ondisk_log_start': "45'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10829824, 'num_objects': 2644, 'num_object_clones': 0, 'num_object_copies': 5288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2644, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14229, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -821}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:18.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:57:18.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'17206", 'reported_seq': 17314, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:14.373427+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:57:14.373427+0000', 'last_peered': '2024-09-16T04:57:14.373427+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:57:14.373427+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'7200", 'ondisk_log_start': "45'7200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10936937, 'num_objects': 2671, 'num_object_clones': 0, 'num_object_copies': 5342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2671, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17205, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -736}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:18.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:57:18.605 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'17816", 'reported_seq': 17929, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:14.428939+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:57:14.428939+0000', 'last_peered': '2024-09-16T04:57:14.428939+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:57:14.428939+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'7800", 'ondisk_log_start': "47'7800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11126587, 'num_objects': 2719, 'num_object_clones': 0, 'num_object_copies': 5438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2719, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17813, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -795}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:18.606 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:57:18.606 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'17727", 'reported_seq': 17877, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:14.589524+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:57:14.589524+0000', 'last_peered': '2024-09-16T04:57:14.589524+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:57:14.589524+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "45'7700", 'ondisk_log_start': "45'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11014761, 'num_objects': 2690, 'num_object_clones': 0, 'num_object_copies': 5380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14496, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:18.606 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:57:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:19.646 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30165/85266 objects degraded (35.378%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":587,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":42633,"data_bytes":174588170,"bytes_used":1761742848,"bytes_avail":288148549632,"bytes_total":289910292480,"degraded_objects":30165,"degraded_total":85266,"degraded_ratio":0.35377524452888609,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":389},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:57:19.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:57:19.676 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:20.467 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:20.467 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:20.556 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-16T04:57:18.484786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":174588170,"num_objects":42633,"num_object_clones":0,"num_object_copies":85266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":244001,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10578,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160724,"ondisk_log_size":160724,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1720452,"kb_used_data":507840,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281395068,"statfs":{"total":289910292480,"available":288148549632,"internally_reserved":0,"allocated":520028160,"data_stored":519095627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1241631745},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,6,6,10,18,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19156992,"num_objects":-4677,"num_object_clones":0,"num_object_copies":-9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4677,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010183"},"pg_stats":[{"pgid":"2.f","version":"51'17139","reported_seq":18423,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.733284+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:14.733284+0000","last_peered":"2024-09-16T04:57:14.733284+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:14.733284+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'7100","ondisk_log_start":"45'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10645504,"num_objects":2599,"num_object_clones":0,"num_object_copies":5198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2744,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17139,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":598,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-145}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17351","reported_seq":17477,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.469335+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:14.469335+0000","last_peered":"2024-09-16T04:57:14.469335+0000","last_clean":"2024-09-16T04:57:14.469335+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:14.469335+0000","last_undegraded":"2024-09-16T04:57:14.469335+0000","last_fullsized":"2024-09-16T04:57:14.469335+0000","mapping_epoch":35,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10825728,"num_objects":2643,"num_object_clones":0,"num_object_copies":5286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14320,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17239","reported_seq":17388,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.411104+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:14.411104+0000","last_peered":"2024-09-16T04:57:14.411104+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:14.411104+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10723352,"num_objects":2619,"num_object_clones":0,"num_object_copies":5238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2619,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13978,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-700}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'17596","reported_seq":17676,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.378473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:14.378473+0000","last_peered":"2024-09-16T04:57:14.378473+0000","last_clean":"2024-09-16T04:57:14.378473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:14.378473+0000","last_undegraded":"2024-09-16T04:57:14.378473+0000","last_fullsized":"2024-09-16T04:57:14.378473+0000","mapping_epoch":35,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11255808,"num_objects":2748,"num_object_clones":0,"num_object_copies":5496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17596,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17510","reported_seq":17739,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.637045+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:14.637045+0000","last_peered":"2024-09-16T04:57:14.637045+0000","last_clean":"2024-09-16T04:57:14.637045+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:14.637045+0000","last_undegraded":"2024-09-16T04:57:14.637045+0000","last_fullsized":"2024-09-16T04:57:14.637045+0000","mapping_epoch":34,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10788864,"num_objects":2634,"num_object_clones":0,"num_object_copies":5268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14542,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17605","reported_seq":17749,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.725162+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:14.725162+0000","last_peered":"2024-09-16T04:57:14.725162+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:14.725162+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"45'7600","ondisk_log_start":"45'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11091968,"num_objects":2708,"num_object_clones":0,"num_object_copies":5416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14598,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17244","reported_seq":17403,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.713213+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:14.713213+0000","last_peered":"2024-09-16T04:57:14.713213+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:14.713213+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10715136,"num_objects":2616,"num_object_clones":0,"num_object_copies":5232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13899,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-783}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17375","reported_seq":17601,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.712689+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:14.712689+0000","last_peered":"2024-09-16T04:57:14.712689+0000","last_clean":"2024-09-16T04:57:14.712689+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:14.712689+0000","last_undegraded":"2024-09-16T04:57:14.712689+0000","last_fullsized":"2024-09-16T04:57:14.712689+0000","mapping_epoch":34,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10817536,"num_objects":2641,"num_object_clones":0,"num_object_copies":5282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14445,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17399","reported_seq":17634,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.382628+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:14.382628+0000","last_peered":"2024-09-16T04:57:14.382628+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:14.382628+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10985472,"num_objects":2682,"num_object_clones":0,"num_object_copies":5364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14150,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-639}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17574","reported_seq":17714,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.732779+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:14.732779+0000","last_peered":"2024-09-16T04:57:14.732779+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:14.732779+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10829824,"num_objects":2644,"num_object_clones":0,"num_object_copies":5288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14229,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-821}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17282","reported_seq":24137,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.420091+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:14.420091+0000","last_peered":"2024-09-16T04:57:14.420091+0000","last_clean":"2024-09-16T04:57:14.420091+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:14.420091+0000","last_undegraded":"2024-09-16T04:57:14.420091+0000","last_fullsized":"2024-09-16T04:57:14.420091+0000","mapping_epoch":50,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:15.063964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11048146,"num_objects":2699,"num_object_clones":0,"num_object_copies":5398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17254,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17351","reported_seq":24218,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.445071+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:14.445071+0000","last_peered":"2024-09-16T04:57:14.445071+0000","last_clean":"2024-09-16T04:57:14.445071+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:14.445071+0000","last_undegraded":"2024-09-16T04:57:14.445071+0000","last_fullsized":"2024-09-16T04:57:14.445071+0000","mapping_epoch":46,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10760192,"num_objects":2627,"num_object_clones":0,"num_object_copies":5254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14074,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17206","reported_seq":17314,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.373427+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:14.373427+0000","last_peered":"2024-09-16T04:57:14.373427+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:14.373427+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10936937,"num_objects":2671,"num_object_clones":0,"num_object_copies":5342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17205,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-736}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17816","reported_seq":17929,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.428939+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:14.428939+0000","last_peered":"2024-09-16T04:57:14.428939+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:14.428939+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11126587,"num_objects":2719,"num_object_clones":0,"num_object_copies":5438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17813,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-795}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17507","reported_seq":24311,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:14.092873+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:14.092873+0000","last_peered":"2024-09-16T04:57:14.092873+0000","last_clean":"2024-09-16T04:57:14.092873+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:14.092873+0000","last_undegraded":"2024-09-16T04:57:14.092873+0000","last_fullsized":"2024-09-16T04:57:14.092873+0000","mapping_epoch":48,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11022336,"num_objects":2691,"num_object_clones":0,"num_object_copies":5382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14261,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17727","reported_seq":17877,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:14.589524+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:14.589524+0000","last_peered":"2024-09-16T04:57:14.589524+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:14.589524+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"45'7700","ondisk_log_start":"45'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11014761,"num_objects":2690,"num_object_clones":0,"num_object_copies":5380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14496,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":174588151,"num_objects":42631,"num_object_clones":0,"num_object_copies":85262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42631,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":243999,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10577,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516497408,"data_stored":516423163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568572,"kb_used_data":189804,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93803268,"statfs":{"total":96636764160,"available":96054546432,"internally_reserved":0,"allocated":194359296,"data_stored":194065649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":387837609},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,2,3,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593820,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571356,"kb_used_data":130116,"kb_used_omap":4,"kb_used_meta":441211,"kb_avail":93800484,"statfs":{"total":96636764160,"available":96051695616,"internally_reserved":0,"allocated":133238784,"data_stored":132889473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":451800750},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580524,"kb_used_data":187920,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93791316,"statfs":{"total":96636764160,"available":96042307584,"internally_reserved":0,"allocated":192430080,"data_stored":192140505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,2,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":191586304,"data_stored":191582825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131604480,"data_stored":131541265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":193306624,"data_stored":193299073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:23.558 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:24.323 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:24.324 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:24.391 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-16T04:57:22.485741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":164954378,"num_objects":40281,"num_object_clones":0,"num_object_copies":80562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246353,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10669,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160976,"ondisk_log_size":160976,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1715484,"kb_used_data":488996,"kb_used_omap":12,"kb_used_meta":1226419,"kb_avail":281400036,"statfs":{"total":289910292480,"available":288153636864,"internally_reserved":0,"allocated":500731904,"data_stored":499799371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1255853057},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,7,6,10,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19124224,"num_objects":-4669,"num_object_clones":0,"num_object_copies":-9338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4669,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008338"},"pg_stats":[{"pgid":"2.f","version":"51'17257","reported_seq":18723,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.757827+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:19.757827+0000","last_peered":"2024-09-16T04:57:19.757827+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:19.757827+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10162176,"num_objects":2481,"num_object_clones":0,"num_object_copies":4962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2653,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17257,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":689,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-172}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17499","reported_seq":17625,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:19.323970+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:19.323970+0000","last_peered":"2024-09-16T04:57:19.323970+0000","last_clean":"2024-09-16T04:57:19.323970+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:19.323970+0000","last_undegraded":"2024-09-16T04:57:19.323970+0000","last_fullsized":"2024-09-16T04:57:19.323970+0000","mapping_epoch":35,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10219520,"num_objects":2495,"num_object_clones":0,"num_object_copies":4990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14468,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17386","reported_seq":17535,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.401522+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:19.401522+0000","last_peered":"2024-09-16T04:57:19.401522+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:19.401522+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10121240,"num_objects":2472,"num_object_clones":0,"num_object_copies":4944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2472,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14125,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-847}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'17761","reported_seq":17841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:19.463938+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:19.463938+0000","last_peered":"2024-09-16T04:57:19.463938+0000","last_clean":"2024-09-16T04:57:19.463938+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:19.463938+0000","last_undegraded":"2024-09-16T04:57:19.463938+0000","last_fullsized":"2024-09-16T04:57:19.463938+0000","mapping_epoch":35,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10579968,"num_objects":2583,"num_object_clones":0,"num_object_copies":5166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17761,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17653","reported_seq":17882,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:19.684361+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:19.684361+0000","last_peered":"2024-09-16T04:57:19.684361+0000","last_clean":"2024-09-16T04:57:19.684361+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:19.684361+0000","last_undegraded":"2024-09-16T04:57:19.684361+0000","last_fullsized":"2024-09-16T04:57:19.684361+0000","mapping_epoch":34,"log_start":"45'7600","ondisk_log_start":"45'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10203136,"num_objects":2491,"num_object_clones":0,"num_object_copies":4982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14685,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17754","reported_seq":17894,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.740109+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:19.740109+0000","last_peered":"2024-09-16T04:57:19.740109+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:19.740109+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10481664,"num_objects":2559,"num_object_clones":0,"num_object_copies":5118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14747,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-897}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17390","reported_seq":17549,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.615715+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:19.615715+0000","last_peered":"2024-09-16T04:57:19.615715+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:19.615715+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10117120,"num_objects":2470,"num_object_clones":0,"num_object_copies":4940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14045,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-929}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17515","reported_seq":17741,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:19.680273+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:19.680273+0000","last_peered":"2024-09-16T04:57:19.680273+0000","last_clean":"2024-09-16T04:57:19.680273+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:19.680273+0000","last_undegraded":"2024-09-16T04:57:19.680273+0000","last_fullsized":"2024-09-16T04:57:19.680273+0000","mapping_epoch":34,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10244096,"num_objects":2501,"num_object_clones":0,"num_object_copies":5002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14585,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17578","reported_seq":17815,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.488511+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:19.488511+0000","last_peered":"2024-09-16T04:57:19.488511+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:19.488511+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10252288,"num_objects":2503,"num_object_clones":0,"num_object_copies":5006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14329,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-818}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17696","reported_seq":17833,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.720118+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:19.720118+0000","last_peered":"2024-09-16T04:57:19.720118+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:19.720118+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'7600","ondisk_log_start":"45'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10330112,"num_objects":2522,"num_object_clones":0,"num_object_copies":5044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14351,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17435","reported_seq":24290,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:19.454154+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:19.454154+0000","last_peered":"2024-09-16T04:57:19.454154+0000","last_clean":"2024-09-16T04:57:19.454154+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:19.454154+0000","last_undegraded":"2024-09-16T04:57:19.454154+0000","last_fullsized":"2024-09-16T04:57:19.454154+0000","mapping_epoch":50,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:21.014847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10421458,"num_objects":2546,"num_object_clones":0,"num_object_copies":5092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17407,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17505","reported_seq":24372,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:19.466955+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:19.466955+0000","last_peered":"2024-09-16T04:57:19.466955+0000","last_clean":"2024-09-16T04:57:19.466955+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:19.466955+0000","last_undegraded":"2024-09-16T04:57:19.466955+0000","last_fullsized":"2024-09-16T04:57:19.466955+0000","mapping_epoch":46,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10129408,"num_objects":2473,"num_object_clones":0,"num_object_copies":4946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14228,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17356","reported_seq":17464,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.480794+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:19.480794+0000","last_peered":"2024-09-16T04:57:19.480794+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:19.480794+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10322537,"num_objects":2521,"num_object_clones":0,"num_object_copies":5042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17355,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-886}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17950","reported_seq":18063,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.415955+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:19.415955+0000","last_peered":"2024-09-16T04:57:19.415955+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:19.415955+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10577723,"num_objects":2585,"num_object_clones":0,"num_object_copies":5170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17947,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-929}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17655","reported_seq":24459,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:19.127551+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:19.127551+0000","last_peered":"2024-09-16T04:57:19.127551+0000","last_clean":"2024-09-16T04:57:19.127551+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:19.127551+0000","last_undegraded":"2024-09-16T04:57:19.127551+0000","last_fullsized":"2024-09-16T04:57:19.127551+0000","mapping_epoch":48,"log_start":"45'7600","ondisk_log_start":"45'7600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10416128,"num_objects":2543,"num_object_clones":0,"num_object_copies":5086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14409,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17883","reported_seq":18033,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.658140+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:19.658140+0000","last_peered":"2024-09-16T04:57:19.658140+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:19.658140+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10375785,"num_objects":2534,"num_object_clones":0,"num_object_copies":5068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14652,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-990}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":164954359,"num_objects":40279,"num_object_clones":0,"num_object_copies":80558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40279,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246351,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10668,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":497147904,"data_stored":497073659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160973,"ondisk_log_size":160973,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584760,"kb_used_data":183784,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93787080,"statfs":{"total":96636764160,"available":96037969920,"internally_reserved":0,"allocated":188194816,"data_stored":187901169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":410578601},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,2,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593821,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":551880,"kb_used_data":123124,"kb_used_omap":4,"kb_used_meta":428731,"kb_avail":93819960,"statfs":{"total":96636764160,"available":96071639040,"internally_reserved":0,"allocated":126078976,"data_stored":125729665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":439021230},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,0,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578844,"kb_used_data":182088,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93792996,"statfs":{"total":96636764160,"available":96044027904,"internally_reserved":0,"allocated":186458112,"data_stored":186168537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":185720832,"data_stored":185717353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":124342272,"data_stored":124279057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":187084800,"data_stored":187077249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:24.393 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:25.193 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:25.194 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:25.261 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-16T04:57:24.486156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":164352266,"num_objects":40134,"num_object_clones":0,"num_object_copies":80268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40134,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246500,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10669,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160923,"ondisk_log_size":160923,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1716616,"kb_used_data":481812,"kb_used_omap":12,"kb_used_meta":1234739,"kb_avail":281398904,"statfs":{"total":289910292480,"available":288152477696,"internally_reserved":0,"allocated":493375488,"data_stored":492442955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1264372737},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,8,9,9,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19726336,"num_objects":-4816,"num_object_clones":0,"num_object_copies":-9632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-183,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4816,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008243"},"pg_stats":[{"pgid":"2.f","version":"51'17257","reported_seq":18723,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.757827+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:19.757827+0000","last_peered":"2024-09-16T04:57:19.757827+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:19.757827+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'7200","ondisk_log_start":"45'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10162176,"num_objects":2481,"num_object_clones":0,"num_object_copies":4962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2653,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17257,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":689,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-172}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17499","reported_seq":17625,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:19.323970+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:19.323970+0000","last_peered":"2024-09-16T04:57:19.323970+0000","last_clean":"2024-09-16T04:57:19.323970+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:19.323970+0000","last_undegraded":"2024-09-16T04:57:19.323970+0000","last_fullsized":"2024-09-16T04:57:19.323970+0000","mapping_epoch":35,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10219520,"num_objects":2495,"num_object_clones":0,"num_object_copies":4990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14468,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17386","reported_seq":17535,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.401522+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:19.401522+0000","last_peered":"2024-09-16T04:57:19.401522+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:19.401522+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10121240,"num_objects":2472,"num_object_clones":0,"num_object_copies":4944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2472,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14125,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-847}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'17761","reported_seq":17841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:19.463938+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:19.463938+0000","last_peered":"2024-09-16T04:57:19.463938+0000","last_clean":"2024-09-16T04:57:19.463938+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:19.463938+0000","last_undegraded":"2024-09-16T04:57:19.463938+0000","last_fullsized":"2024-09-16T04:57:19.463938+0000","mapping_epoch":35,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10579968,"num_objects":2583,"num_object_clones":0,"num_object_copies":5166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17761,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17653","reported_seq":17882,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:19.684361+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:19.684361+0000","last_peered":"2024-09-16T04:57:19.684361+0000","last_clean":"2024-09-16T04:57:19.684361+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:19.684361+0000","last_undegraded":"2024-09-16T04:57:19.684361+0000","last_fullsized":"2024-09-16T04:57:19.684361+0000","mapping_epoch":34,"log_start":"45'7600","ondisk_log_start":"45'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10203136,"num_objects":2491,"num_object_clones":0,"num_object_copies":4982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14685,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17754","reported_seq":17894,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.740109+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:19.740109+0000","last_peered":"2024-09-16T04:57:19.740109+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:19.740109+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10481664,"num_objects":2559,"num_object_clones":0,"num_object_copies":5118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14747,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-897}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17390","reported_seq":17549,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.615715+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:19.615715+0000","last_peered":"2024-09-16T04:57:19.615715+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:19.615715+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10117120,"num_objects":2470,"num_object_clones":0,"num_object_copies":4940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14045,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-929}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17515","reported_seq":17741,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:19.680273+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:19.680273+0000","last_peered":"2024-09-16T04:57:19.680273+0000","last_clean":"2024-09-16T04:57:19.680273+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:19.680273+0000","last_undegraded":"2024-09-16T04:57:19.680273+0000","last_fullsized":"2024-09-16T04:57:19.680273+0000","mapping_epoch":34,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10244096,"num_objects":2501,"num_object_clones":0,"num_object_copies":5002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14585,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17578","reported_seq":17815,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.488511+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:19.488511+0000","last_peered":"2024-09-16T04:57:19.488511+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:19.488511+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10252288,"num_objects":2503,"num_object_clones":0,"num_object_copies":5006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14329,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-818}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17696","reported_seq":17833,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.720118+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:19.720118+0000","last_peered":"2024-09-16T04:57:19.720118+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:19.720118+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"45'7600","ondisk_log_start":"45'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10330112,"num_objects":2522,"num_object_clones":0,"num_object_copies":5044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14351,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17435","reported_seq":24290,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:19.454154+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:19.454154+0000","last_peered":"2024-09-16T04:57:19.454154+0000","last_clean":"2024-09-16T04:57:19.454154+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:19.454154+0000","last_undegraded":"2024-09-16T04:57:19.454154+0000","last_fullsized":"2024-09-16T04:57:19.454154+0000","mapping_epoch":50,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:21.014847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10421458,"num_objects":2546,"num_object_clones":0,"num_object_copies":5092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17407,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17505","reported_seq":24372,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:19.466955+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:19.466955+0000","last_peered":"2024-09-16T04:57:19.466955+0000","last_clean":"2024-09-16T04:57:19.466955+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:19.466955+0000","last_undegraded":"2024-09-16T04:57:19.466955+0000","last_fullsized":"2024-09-16T04:57:19.466955+0000","mapping_epoch":46,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10129408,"num_objects":2473,"num_object_clones":0,"num_object_copies":4946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14228,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17356","reported_seq":17464,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.480794+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:19.480794+0000","last_peered":"2024-09-16T04:57:19.480794+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:19.480794+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"45'7300","ondisk_log_start":"45'7300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10322537,"num_objects":2521,"num_object_clones":0,"num_object_copies":5042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17355,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-886}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'17950","reported_seq":18063,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.415955+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:19.415955+0000","last_peered":"2024-09-16T04:57:19.415955+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:19.415955+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10577723,"num_objects":2585,"num_object_clones":0,"num_object_copies":5170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17947,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-929}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17802","reported_seq":24606,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:24.130505+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:24.130505+0000","last_peered":"2024-09-16T04:57:24.130505+0000","last_clean":"2024-09-16T04:57:24.130505+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:24.130505+0000","last_undegraded":"2024-09-16T04:57:24.130505+0000","last_fullsized":"2024-09-16T04:57:24.130505+0000","mapping_epoch":48,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9814016,"num_objects":2396,"num_object_clones":0,"num_object_copies":4792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14556,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'17883","reported_seq":18033,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:19.658140+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:19.658140+0000","last_peered":"2024-09-16T04:57:19.658140+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:19.658140+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10375785,"num_objects":2534,"num_object_clones":0,"num_object_copies":5068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14652,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-990}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":164352247,"num_objects":40132,"num_object_clones":0,"num_object_copies":80264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40132,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246498,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10668,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":489869312,"data_stored":489795067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160920,"ondisk_log_size":160920,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584760,"kb_used_data":183784,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93787080,"statfs":{"total":96636764160,"available":96037969920,"internally_reserved":0,"allocated":188194816,"data_stored":187901169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":410578601},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,2,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593822,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553012,"kb_used_data":115940,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93818828,"statfs":{"total":96636764160,"available":96070479872,"internally_reserved":0,"allocated":118722560,"data_stored":118373249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":447540910},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578844,"kb_used_data":182088,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93792996,"statfs":{"total":96636764160,"available":96044027904,"internally_reserved":0,"allocated":186458112,"data_stored":186168537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":185720832,"data_stored":185717353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":117063680,"data_stored":117000465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":187084800,"data_stored":187077249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:25.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:57:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'17257", 'reported_seq': 18723, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:19.757827+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:57:19.757827+0000', 'last_peered': '2024-09-16T04:57:19.757827+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:57:19.757827+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "45'7200", 'ondisk_log_start': "45'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10162176, 'num_objects': 2481, 'num_object_clones': 0, 'num_object_copies': 4962, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2653, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2481, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17257, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 689, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -172}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:57:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'17386", 'reported_seq': 17535, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:19.401522+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:57:19.401522+0000', 'last_peered': '2024-09-16T04:57:19.401522+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:57:19.401522+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "45'7300", 'ondisk_log_start': "45'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10121240, 'num_objects': 2472, 'num_object_clones': 0, 'num_object_copies': 4944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2472, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14125, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -847}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:57:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'17754", 'reported_seq': 17894, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:19.740109+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:57:19.740109+0000', 'last_peered': '2024-09-16T04:57:19.740109+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:57:19.740109+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'7700", 'ondisk_log_start': "47'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10481664, 'num_objects': 2559, 'num_object_clones': 0, 'num_object_copies': 5118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2559, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14747, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -897}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:57:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'17390", 'reported_seq': 17549, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:19.615715+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:57:19.615715+0000', 'last_peered': '2024-09-16T04:57:19.615715+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:57:19.615715+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "45'7300", 'ondisk_log_start': "45'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10117120, 'num_objects': 2470, 'num_object_clones': 0, 'num_object_copies': 4940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2470, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14045, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -929}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:57:25.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'17578", 'reported_seq': 17815, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:19.488511+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:57:19.488511+0000', 'last_peered': '2024-09-16T04:57:19.488511+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:57:19.488511+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "45'7500", 'ondisk_log_start': "45'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10252288, 'num_objects': 2503, 'num_object_clones': 0, 'num_object_copies': 5006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2503, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14329, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -818}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:25.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:57:25.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'17696", 'reported_seq': 17833, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:19.720118+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:57:19.720118+0000', 'last_peered': '2024-09-16T04:57:19.720118+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:57:19.720118+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "45'7600", 'ondisk_log_start': "45'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10330112, 'num_objects': 2522, 'num_object_clones': 0, 'num_object_copies': 5044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2522, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14351, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:25.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:57:25.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'17356", 'reported_seq': 17464, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:19.480794+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:57:19.480794+0000', 'last_peered': '2024-09-16T04:57:19.480794+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:57:19.480794+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "45'7300", 'ondisk_log_start': "45'7300", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10322537, 'num_objects': 2521, 'num_object_clones': 0, 'num_object_copies': 5042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2521, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17355, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -886}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:25.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:57:25.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'17950", 'reported_seq': 18063, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:19.415955+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:57:19.415955+0000', 'last_peered': '2024-09-16T04:57:19.415955+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:57:19.415955+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'7900", 'ondisk_log_start': "47'7900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10577723, 'num_objects': 2585, 'num_object_clones': 0, 'num_object_copies': 5170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2585, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17947, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -929}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:25.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:57:25.264 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'17883", 'reported_seq': 18033, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:19.658140+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:57:19.658140+0000', 'last_peered': '2024-09-16T04:57:19.658140+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:57:19.658140+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'7800", 'ondisk_log_start': "47'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10375785, 'num_objects': 2534, 'num_object_clones': 0, 'num_object_copies': 5068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2534, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14652, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -990}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:25.265 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:57:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:26.241 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30074/80268 objects degraded (37.467%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":594,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":40134,"data_bytes":164352266,"bytes_used":1757814784,"bytes_avail":288152477696,"bytes_total":289910292480,"degraded_objects":30074,"degraded_total":80268,"degraded_ratio":0.37466985598245878,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":183,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":401},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:57:26.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:57:26.328 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:27.326 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:27.326 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:27.402 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-16T04:57:26.486604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":155246858,"num_objects":37911,"num_object_clones":0,"num_object_copies":75822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37911,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248723,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10754,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160646,"ondisk_log_size":160646,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1700868,"kb_used_data":470224,"kb_used_omap":12,"kb_used_meta":1230579,"kb_avail":281414652,"statfs":{"total":289910292480,"available":288168603648,"internally_reserved":0,"allocated":481509376,"data_stored":480576843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1260112897},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,6,6,9,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28241920,"num_objects":-6895,"num_object_clones":0,"num_object_copies":-13790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6895,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005961"},"pg_stats":[{"pgid":"2.f","version":"51'17403","reported_seq":19040,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:24.744271+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:24.744271+0000","last_peered":"2024-09-16T04:57:24.744271+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:24.744271+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"45'7400","ondisk_log_start":"45'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9564160,"num_objects":2335,"num_object_clones":0,"num_object_copies":4670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2568,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17403,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":774,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-233}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17651","reported_seq":17777,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:24.551347+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:24.551347+0000","last_peered":"2024-09-16T04:57:24.551347+0000","last_clean":"2024-09-16T04:57:24.551347+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:24.551347+0000","last_undegraded":"2024-09-16T04:57:24.551347+0000","last_fullsized":"2024-09-16T04:57:24.551347+0000","mapping_epoch":35,"log_start":"47'7600","ondisk_log_start":"47'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9596928,"num_objects":2343,"num_object_clones":0,"num_object_copies":4686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14620,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17528","reported_seq":17677,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:24.496684+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:24.496684+0000","last_peered":"2024-09-16T04:57:24.496684+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:24.496684+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"45'7500","ondisk_log_start":"45'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9539608,"num_objects":2330,"num_object_clones":0,"num_object_copies":4660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2330,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14267,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-989}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'17918","reported_seq":17998,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:24.488631+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:24.488631+0000","last_peered":"2024-09-16T04:57:24.488631+0000","last_clean":"2024-09-16T04:57:24.488631+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:24.488631+0000","last_undegraded":"2024-09-16T04:57:24.488631+0000","last_fullsized":"2024-09-16T04:57:24.488631+0000","mapping_epoch":35,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9936896,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17918,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17794","reported_seq":18022,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:24.719469+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:24.719469+0000","last_peered":"2024-09-16T04:57:24.719469+0000","last_clean":"2024-09-16T04:57:24.719469+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:24.719469+0000","last_undegraded":"2024-09-16T04:57:24.719469+0000","last_fullsized":"2024-09-16T04:57:24.719469+0000","mapping_epoch":34,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9625600,"num_objects":2350,"num_object_clones":0,"num_object_copies":4700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14826,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'17916","reported_seq":18057,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:24.739097+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:24.739097+0000","last_peered":"2024-09-16T04:57:24.739097+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:24.739097+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9818112,"num_objects":2397,"num_object_clones":0,"num_object_copies":4794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14909,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1059}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17550","reported_seq":17709,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:24.705178+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:24.705178+0000","last_peered":"2024-09-16T04:57:24.705178+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:24.705178+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"46'7500","ondisk_log_start":"46'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9461760,"num_objects":2310,"num_object_clones":0,"num_object_copies":4620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14205,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1089}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17653","reported_seq":17879,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:24.683858+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:24.683858+0000","last_peered":"2024-09-16T04:57:24.683858+0000","last_clean":"2024-09-16T04:57:24.683858+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:24.683858+0000","last_undegraded":"2024-09-16T04:57:24.683858+0000","last_fullsized":"2024-09-16T04:57:24.683858+0000","mapping_epoch":34,"log_start":"47'7600","ondisk_log_start":"47'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9678848,"num_objects":2363,"num_object_clones":0,"num_object_copies":4726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14723,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17731","reported_seq":17968,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:24.554034+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:24.554034+0000","last_peered":"2024-09-16T04:57:24.554034+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:24.554034+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9625600,"num_objects":2350,"num_object_clones":0,"num_object_copies":4700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14482,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-971}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17845","reported_seq":17982,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:24.760788+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:24.760788+0000","last_peered":"2024-09-16T04:57:24.760788+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:24.760788+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9719808,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14500,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1092}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17575","reported_seq":24430,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:24.518458+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:24.518458+0000","last_peered":"2024-09-16T04:57:24.518458+0000","last_clean":"2024-09-16T04:57:24.518458+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:24.518458+0000","last_undegraded":"2024-09-16T04:57:24.518458+0000","last_fullsized":"2024-09-16T04:57:24.518458+0000","mapping_epoch":50,"log_start":"46'7500","ondisk_log_start":"46'7500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:26.920098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9848018,"num_objects":2406,"num_object_clones":0,"num_object_copies":4812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17547,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17638","reported_seq":24505,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:24.371561+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:24.371561+0000","last_peered":"2024-09-16T04:57:24.371561+0000","last_clean":"2024-09-16T04:57:24.371561+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:24.371561+0000","last_undegraded":"2024-09-16T04:57:24.371561+0000","last_fullsized":"2024-09-16T04:57:24.371561+0000","mapping_epoch":46,"log_start":"47'7600","ondisk_log_start":"47'7600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9584640,"num_objects":2340,"num_object_clones":0,"num_object_copies":4680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14361,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17516","reported_seq":17624,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:24.457156+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:24.457156+0000","last_peered":"2024-09-16T04:57:24.457156+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:24.457156+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'7500","ondisk_log_start":"47'7500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9667177,"num_objects":2361,"num_object_clones":0,"num_object_copies":4722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17515,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1046}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18098","reported_seq":18211,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:24.409381+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:24.409381+0000","last_peered":"2024-09-16T04:57:24.409381+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:24.409381+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9971515,"num_objects":2437,"num_object_clones":0,"num_object_copies":4874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18095,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1077}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17802","reported_seq":24606,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:24.130505+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:24.130505+0000","last_peered":"2024-09-16T04:57:24.130505+0000","last_clean":"2024-09-16T04:57:24.130505+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:24.130505+0000","last_undegraded":"2024-09-16T04:57:24.130505+0000","last_fullsized":"2024-09-16T04:57:24.130505+0000","mapping_epoch":48,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9814016,"num_objects":2396,"num_object_clones":0,"num_object_copies":4792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14556,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18025","reported_seq":18175,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:24.700619+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:24.700619+0000","last_peered":"2024-09-16T04:57:24.700619+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:24.700619+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9794153,"num_objects":2392,"num_object_clones":0,"num_object_copies":4784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14794,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1132}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":155246839,"num_objects":37909,"num_object_clones":0,"num_object_copies":75818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":248721,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10753,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":477782016,"data_stored":477707771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160643,"ondisk_log_size":160643,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566576,"kb_used_data":178080,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93805264,"statfs":{"total":96636764160,"available":96056590336,"internally_reserved":0,"allocated":182353920,"data_stored":182060273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":397799081},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593822,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553012,"kb_used_data":115940,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93818828,"statfs":{"total":96636764160,"available":96070479872,"internally_reserved":0,"allocated":118722560,"data_stored":118373249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":447540910},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581280,"kb_used_data":176204,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93790560,"statfs":{"total":96636764160,"available":96041533440,"internally_reserved":0,"allocated":180432896,"data_stored":180143321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":414772906},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,3,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":179605504,"data_stored":179602025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":117063680,"data_stored":117000465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":181112832,"data_stored":181105281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:30.404 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:31.262 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:31.263 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:31.335 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-16T04:57:30.487323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":145555722,"num_objects":35545,"num_object_clones":0,"num_object_copies":71090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251089,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10831,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1665532,"kb_used_data":451520,"kb_used_omap":12,"kb_used_meta":1213939,"kb_avail":281449988,"statfs":{"total":289910292480,"available":288204787712,"internally_reserved":0,"allocated":462356480,"data_stored":461423947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1243073537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,8,9,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29032448,"num_objects":-7088,"num_object_clones":0,"num_object_copies":-14176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-253,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7088,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002536"},"pg_stats":[{"pgid":"2.f","version":"51'17548","reported_seq":19339,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.755031+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:29.755031+0000","last_peered":"2024-09-16T04:57:29.755031+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:29.755031+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'7500","ondisk_log_start":"47'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8970240,"num_objects":2190,"num_object_clones":0,"num_object_copies":4380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2491,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17548,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":851,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-301}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17809","reported_seq":17935,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.540976+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:29.540976+0000","last_peered":"2024-09-16T04:57:29.540976+0000","last_clean":"2024-09-16T04:57:29.540976+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:29.540976+0000","last_undegraded":"2024-09-16T04:57:29.540976+0000","last_fullsized":"2024-09-16T04:57:29.540976+0000","mapping_epoch":35,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8949760,"num_objects":2185,"num_object_clones":0,"num_object_copies":4370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14778,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17668","reported_seq":17817,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.544211+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:29.544211+0000","last_peered":"2024-09-16T04:57:29.544211+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:29.544211+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'7600","ondisk_log_start":"47'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8966168,"num_objects":2190,"num_object_clones":0,"num_object_copies":4380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14407,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1129}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18056","reported_seq":18136,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.512146+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:29.512146+0000","last_peered":"2024-09-16T04:57:29.512146+0000","last_clean":"2024-09-16T04:57:29.512146+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:29.512146+0000","last_undegraded":"2024-09-16T04:57:29.512146+0000","last_fullsized":"2024-09-16T04:57:29.512146+0000","mapping_epoch":35,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9371648,"num_objects":2288,"num_object_clones":0,"num_object_copies":4576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18056,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17930","reported_seq":18159,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.716204+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:29.716204+0000","last_peered":"2024-09-16T04:57:29.716204+0000","last_clean":"2024-09-16T04:57:29.716204+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:29.716204+0000","last_undegraded":"2024-09-16T04:57:29.716204+0000","last_fullsized":"2024-09-16T04:57:29.716204+0000","mapping_epoch":34,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9068544,"num_objects":2214,"num_object_clones":0,"num_object_copies":4428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14962,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18056","reported_seq":18200,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.687421+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:29.687421+0000","last_peered":"2024-09-16T04:57:29.687421+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:29.687421+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9244672,"num_objects":2257,"num_object_clones":0,"num_object_copies":4514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15049,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1199}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17704","reported_seq":17863,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.723620+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:29.723620+0000","last_peered":"2024-09-16T04:57:29.723620+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:29.723620+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8830976,"num_objects":2156,"num_object_clones":0,"num_object_copies":4312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14359,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17813","reported_seq":18039,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.768229+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:29.768229+0000","last_peered":"2024-09-16T04:57:29.768229+0000","last_clean":"2024-09-16T04:57:29.768229+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:29.768229+0000","last_undegraded":"2024-09-16T04:57:29.768229+0000","last_fullsized":"2024-09-16T04:57:29.768229+0000","mapping_epoch":34,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9023488,"num_objects":2203,"num_object_clones":0,"num_object_copies":4406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14883,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17870","reported_seq":18108,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.529236+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:29.529236+0000","last_peered":"2024-09-16T04:57:29.529236+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:29.529236+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9056256,"num_objects":2211,"num_object_clones":0,"num_object_copies":4422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14621,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1110}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17991","reported_seq":18131,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.738780+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:29.738780+0000","last_peered":"2024-09-16T04:57:29.738780+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:29.738780+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9121792,"num_objects":2227,"num_object_clones":0,"num_object_copies":4454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14646,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1238}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17727","reported_seq":24582,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.489992+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:29.489992+0000","last_peered":"2024-09-16T04:57:29.489992+0000","last_clean":"2024-09-16T04:57:29.489992+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:29.489992+0000","last_undegraded":"2024-09-16T04:57:29.489992+0000","last_fullsized":"2024-09-16T04:57:29.489992+0000","mapping_epoch":50,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:32.853641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9225426,"num_objects":2254,"num_object_clones":0,"num_object_copies":4508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17699,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17782","reported_seq":24649,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.564123+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:29.564123+0000","last_peered":"2024-09-16T04:57:29.564123+0000","last_clean":"2024-09-16T04:57:29.564123+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:29.564123+0000","last_undegraded":"2024-09-16T04:57:29.564123+0000","last_fullsized":"2024-09-16T04:57:29.564123+0000","mapping_epoch":46,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8994816,"num_objects":2196,"num_object_clones":0,"num_object_copies":4392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14505,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17691","reported_seq":17797,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.485848+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:29.485848+0000","last_peered":"2024-09-16T04:57:29.485848+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:29.485848+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'7600","ondisk_log_start":"47'7600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8950377,"num_objects":2186,"num_object_clones":0,"num_object_copies":4372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17690,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1221}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18242","reported_seq":18355,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.529357+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:29.529357+0000","last_peered":"2024-09-16T04:57:29.529357+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:29.529357+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9381691,"num_objects":2293,"num_object_clones":0,"num_object_copies":4586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18239,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1221}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17963","reported_seq":24767,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.165320+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:29.165320+0000","last_peered":"2024-09-16T04:57:29.165320+0000","last_clean":"2024-09-16T04:57:29.165320+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:29.165320+0000","last_undegraded":"2024-09-16T04:57:29.165320+0000","last_fullsized":"2024-09-16T04:57:29.165320+0000","mapping_epoch":48,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9154560,"num_objects":2235,"num_object_clones":0,"num_object_copies":4470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14717,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18159","reported_seq":18309,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.710601+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:29.710601+0000","last_peered":"2024-09-16T04:57:29.710601+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:29.710601+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9245289,"num_objects":2258,"num_object_clones":0,"num_object_copies":4516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14928,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":145555703,"num_objects":35543,"num_object_clones":0,"num_object_copies":71086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251087,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10830,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":458297344,"data_stored":458223099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569112,"kb_used_data":172288,"kb_used_omap":4,"kb_used_meta":396795,"kb_avail":93802728,"statfs":{"total":96636764160,"available":96053993472,"internally_reserved":0,"allocated":176422912,"data_stored":176129265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":406318761},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593823,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533460,"kb_used_data":108864,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93838380,"statfs":{"total":96636764160,"available":96090501120,"internally_reserved":0,"allocated":111476736,"data_stored":111127425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":434761390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562960,"kb_used_data":170368,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93808880,"statfs":{"total":96636764160,"available":96060293120,"internally_reserved":0,"allocated":174456832,"data_stored":174167257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":173572096,"data_stored":173568617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109707264,"data_stored":109644049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":175017984,"data_stored":175010433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:31.336 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:32.222 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:32.328 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-16T04:57:30.487323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":145555722,"num_objects":35545,"num_object_clones":0,"num_object_copies":71090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251089,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10831,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1665532,"kb_used_data":451520,"kb_used_omap":12,"kb_used_meta":1213939,"kb_avail":281449988,"statfs":{"total":289910292480,"available":288204787712,"internally_reserved":0,"allocated":462356480,"data_stored":461423947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1243073537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,8,9,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29032448,"num_objects":-7088,"num_object_clones":0,"num_object_copies":-14176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-253,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7088,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002536"},"pg_stats":[{"pgid":"2.f","version":"51'17548","reported_seq":19339,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.755031+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:29.755031+0000","last_peered":"2024-09-16T04:57:29.755031+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:29.755031+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'7500","ondisk_log_start":"47'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8970240,"num_objects":2190,"num_object_clones":0,"num_object_copies":4380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2491,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17548,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":851,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-301}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17809","reported_seq":17935,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.540976+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:29.540976+0000","last_peered":"2024-09-16T04:57:29.540976+0000","last_clean":"2024-09-16T04:57:29.540976+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:29.540976+0000","last_undegraded":"2024-09-16T04:57:29.540976+0000","last_fullsized":"2024-09-16T04:57:29.540976+0000","mapping_epoch":35,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8949760,"num_objects":2185,"num_object_clones":0,"num_object_copies":4370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14778,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17668","reported_seq":17817,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.544211+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:29.544211+0000","last_peered":"2024-09-16T04:57:29.544211+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:29.544211+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'7600","ondisk_log_start":"47'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8966168,"num_objects":2190,"num_object_clones":0,"num_object_copies":4380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14407,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1129}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18056","reported_seq":18136,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.512146+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:29.512146+0000","last_peered":"2024-09-16T04:57:29.512146+0000","last_clean":"2024-09-16T04:57:29.512146+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:29.512146+0000","last_undegraded":"2024-09-16T04:57:29.512146+0000","last_fullsized":"2024-09-16T04:57:29.512146+0000","mapping_epoch":35,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9371648,"num_objects":2288,"num_object_clones":0,"num_object_copies":4576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18056,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17930","reported_seq":18159,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.716204+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:29.716204+0000","last_peered":"2024-09-16T04:57:29.716204+0000","last_clean":"2024-09-16T04:57:29.716204+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:29.716204+0000","last_undegraded":"2024-09-16T04:57:29.716204+0000","last_fullsized":"2024-09-16T04:57:29.716204+0000","mapping_epoch":34,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9068544,"num_objects":2214,"num_object_clones":0,"num_object_copies":4428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14962,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18056","reported_seq":18200,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.687421+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:29.687421+0000","last_peered":"2024-09-16T04:57:29.687421+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:29.687421+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9244672,"num_objects":2257,"num_object_clones":0,"num_object_copies":4514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15049,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1199}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17704","reported_seq":17863,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.723620+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:29.723620+0000","last_peered":"2024-09-16T04:57:29.723620+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:29.723620+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8830976,"num_objects":2156,"num_object_clones":0,"num_object_copies":4312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14359,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17813","reported_seq":18039,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.768229+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:29.768229+0000","last_peered":"2024-09-16T04:57:29.768229+0000","last_clean":"2024-09-16T04:57:29.768229+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:29.768229+0000","last_undegraded":"2024-09-16T04:57:29.768229+0000","last_fullsized":"2024-09-16T04:57:29.768229+0000","mapping_epoch":34,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9023488,"num_objects":2203,"num_object_clones":0,"num_object_copies":4406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14883,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17870","reported_seq":18108,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.529236+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:29.529236+0000","last_peered":"2024-09-16T04:57:29.529236+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:29.529236+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9056256,"num_objects":2211,"num_object_clones":0,"num_object_copies":4422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14621,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1110}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17991","reported_seq":18131,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.738780+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:29.738780+0000","last_peered":"2024-09-16T04:57:29.738780+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:29.738780+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9121792,"num_objects":2227,"num_object_clones":0,"num_object_copies":4454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14646,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1238}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17727","reported_seq":24582,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.489992+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:29.489992+0000","last_peered":"2024-09-16T04:57:29.489992+0000","last_clean":"2024-09-16T04:57:29.489992+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:29.489992+0000","last_undegraded":"2024-09-16T04:57:29.489992+0000","last_fullsized":"2024-09-16T04:57:29.489992+0000","mapping_epoch":50,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:32.853641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9225426,"num_objects":2254,"num_object_clones":0,"num_object_copies":4508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17699,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17782","reported_seq":24649,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.564123+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:29.564123+0000","last_peered":"2024-09-16T04:57:29.564123+0000","last_clean":"2024-09-16T04:57:29.564123+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:29.564123+0000","last_undegraded":"2024-09-16T04:57:29.564123+0000","last_fullsized":"2024-09-16T04:57:29.564123+0000","mapping_epoch":46,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8994816,"num_objects":2196,"num_object_clones":0,"num_object_copies":4392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14505,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17691","reported_seq":17797,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.485848+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:29.485848+0000","last_peered":"2024-09-16T04:57:29.485848+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:29.485848+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'7600","ondisk_log_start":"47'7600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8950377,"num_objects":2186,"num_object_clones":0,"num_object_copies":4372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17690,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1221}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18242","reported_seq":18355,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.529357+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:29.529357+0000","last_peered":"2024-09-16T04:57:29.529357+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:29.529357+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9381691,"num_objects":2293,"num_object_clones":0,"num_object_copies":4586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18239,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1221}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17963","reported_seq":24767,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.165320+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:29.165320+0000","last_peered":"2024-09-16T04:57:29.165320+0000","last_clean":"2024-09-16T04:57:29.165320+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:29.165320+0000","last_undegraded":"2024-09-16T04:57:29.165320+0000","last_fullsized":"2024-09-16T04:57:29.165320+0000","mapping_epoch":48,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9154560,"num_objects":2235,"num_object_clones":0,"num_object_copies":4470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14717,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18159","reported_seq":18309,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.710601+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:29.710601+0000","last_peered":"2024-09-16T04:57:29.710601+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:29.710601+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9245289,"num_objects":2258,"num_object_clones":0,"num_object_copies":4516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14928,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":145555703,"num_objects":35543,"num_object_clones":0,"num_object_copies":71086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251087,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10830,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":458297344,"data_stored":458223099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569112,"kb_used_data":172288,"kb_used_omap":4,"kb_used_meta":396795,"kb_avail":93802728,"statfs":{"total":96636764160,"available":96053993472,"internally_reserved":0,"allocated":176422912,"data_stored":176129265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":406318761},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593823,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533460,"kb_used_data":108864,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93838380,"statfs":{"total":96636764160,"available":96090501120,"internally_reserved":0,"allocated":111476736,"data_stored":111127425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":434761390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562960,"kb_used_data":170368,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93808880,"statfs":{"total":96636764160,"available":96060293120,"internally_reserved":0,"allocated":174456832,"data_stored":174167257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":173572096,"data_stored":173568617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109707264,"data_stored":109644049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":175017984,"data_stored":175010433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:32.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:57:32.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'17548", 'reported_seq': 19339, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:29.755031+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:57:29.755031+0000', 'last_peered': '2024-09-16T04:57:29.755031+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:57:29.755031+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'7500", 'ondisk_log_start': "47'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8970240, 'num_objects': 2190, 'num_object_clones': 0, 'num_object_copies': 4380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2491, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2190, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17548, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 851, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -301}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:32.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:57:32.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'17668", 'reported_seq': 17817, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:29.544211+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:57:29.544211+0000', 'last_peered': '2024-09-16T04:57:29.544211+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:57:29.544211+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'7600", 'ondisk_log_start': "47'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8966168, 'num_objects': 2190, 'num_object_clones': 0, 'num_object_copies': 4380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2190, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14407, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1129}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:32.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:57:32.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'18056", 'reported_seq': 18200, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:29.687421+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:57:29.687421+0000', 'last_peered': '2024-09-16T04:57:29.687421+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:57:29.687421+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'8000", 'ondisk_log_start': "47'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9244672, 'num_objects': 2257, 'num_object_clones': 0, 'num_object_copies': 4514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2257, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15049, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1199}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:32.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:57:32.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'17704", 'reported_seq': 17863, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:29.723620+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:57:29.723620+0000', 'last_peered': '2024-09-16T04:57:29.723620+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:57:29.723620+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'7700", 'ondisk_log_start': "47'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8830976, 'num_objects': 2156, 'num_object_clones': 0, 'num_object_copies': 4312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14359, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1243}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:32.330 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:57:32.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'17870", 'reported_seq': 18108, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:29.529236+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:57:29.529236+0000', 'last_peered': '2024-09-16T04:57:29.529236+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:57:29.529236+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'7800", 'ondisk_log_start': "47'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9056256, 'num_objects': 2211, 'num_object_clones': 0, 'num_object_copies': 4422, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2211, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14621, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1110}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:32.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:57:32.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'17991", 'reported_seq': 18131, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:29.738780+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:57:29.738780+0000', 'last_peered': '2024-09-16T04:57:29.738780+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:57:29.738780+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'7900", 'ondisk_log_start': "47'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9121792, 'num_objects': 2227, 'num_object_clones': 0, 'num_object_copies': 4454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2227, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14646, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1238}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:32.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:57:32.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'17691", 'reported_seq': 17797, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:29.485848+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:57:29.485848+0000', 'last_peered': '2024-09-16T04:57:29.485848+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:57:29.485848+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'7600", 'ondisk_log_start': "47'7600", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8950377, 'num_objects': 2186, 'num_object_clones': 0, 'num_object_copies': 4372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2186, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17690, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1221}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:32.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:57:32.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'18242", 'reported_seq': 18355, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:29.529357+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:57:29.529357+0000', 'last_peered': '2024-09-16T04:57:29.529357+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:57:29.529357+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'8200", 'ondisk_log_start': "47'8200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9381691, 'num_objects': 2293, 'num_object_clones': 0, 'num_object_copies': 4586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2293, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18239, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1221}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:32.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:57:32.331 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'18159", 'reported_seq': 18309, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:29.710601+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:57:29.710601+0000', 'last_peered': '2024-09-16T04:57:29.710601+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:57:29.710601+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'8100", 'ondisk_log_start': "47'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9245289, 'num_objects': 2258, 'num_object_clones': 0, 'num_object_copies': 4516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2258, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14928, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1266}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:32.332 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:57:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:33.296 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29912/71090 objects degraded (42.076%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":601,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":35545,"data_bytes":145555722,"bytes_used":1705504768,"bytes_avail":288204787712,"bytes_total":289910292480,"degraded_objects":29912,"degraded_total":71090,"degraded_ratio":0.42076241384160923,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":162,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":394},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:57:33.355 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:57:33.355 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:34.376 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:34.378 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:34.433 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-16T04:57:32.487785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":145555722,"num_objects":35545,"num_object_clones":0,"num_object_copies":71090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251089,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10831,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1665532,"kb_used_data":451520,"kb_used_omap":12,"kb_used_meta":1213939,"kb_avail":281449988,"statfs":{"total":289910292480,"available":288204787712,"internally_reserved":0,"allocated":462356480,"data_stored":461423947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1243073537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,8,9,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19398656,"num_objects":-4736,"num_object_clones":0,"num_object_copies":-9472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4736,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002571"},"pg_stats":[{"pgid":"2.f","version":"51'17548","reported_seq":19339,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.755031+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:29.755031+0000","last_peered":"2024-09-16T04:57:29.755031+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:29.755031+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'7500","ondisk_log_start":"47'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8970240,"num_objects":2190,"num_object_clones":0,"num_object_copies":4380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2491,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17548,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":851,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-301}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17809","reported_seq":17935,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.540976+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:29.540976+0000","last_peered":"2024-09-16T04:57:29.540976+0000","last_clean":"2024-09-16T04:57:29.540976+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:29.540976+0000","last_undegraded":"2024-09-16T04:57:29.540976+0000","last_fullsized":"2024-09-16T04:57:29.540976+0000","mapping_epoch":35,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8949760,"num_objects":2185,"num_object_clones":0,"num_object_copies":4370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14778,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17668","reported_seq":17817,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.544211+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:29.544211+0000","last_peered":"2024-09-16T04:57:29.544211+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:29.544211+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'7600","ondisk_log_start":"47'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8966168,"num_objects":2190,"num_object_clones":0,"num_object_copies":4380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14407,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1129}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18056","reported_seq":18136,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.512146+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:29.512146+0000","last_peered":"2024-09-16T04:57:29.512146+0000","last_clean":"2024-09-16T04:57:29.512146+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:29.512146+0000","last_undegraded":"2024-09-16T04:57:29.512146+0000","last_fullsized":"2024-09-16T04:57:29.512146+0000","mapping_epoch":35,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9371648,"num_objects":2288,"num_object_clones":0,"num_object_copies":4576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18056,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'17930","reported_seq":18159,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.716204+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:29.716204+0000","last_peered":"2024-09-16T04:57:29.716204+0000","last_clean":"2024-09-16T04:57:29.716204+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:29.716204+0000","last_undegraded":"2024-09-16T04:57:29.716204+0000","last_fullsized":"2024-09-16T04:57:29.716204+0000","mapping_epoch":34,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9068544,"num_objects":2214,"num_object_clones":0,"num_object_copies":4428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14962,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18056","reported_seq":18200,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.687421+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:29.687421+0000","last_peered":"2024-09-16T04:57:29.687421+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:29.687421+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9244672,"num_objects":2257,"num_object_clones":0,"num_object_copies":4514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15049,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1199}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17704","reported_seq":17863,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.723620+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:29.723620+0000","last_peered":"2024-09-16T04:57:29.723620+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:29.723620+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8830976,"num_objects":2156,"num_object_clones":0,"num_object_copies":4312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14359,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17813","reported_seq":18039,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.768229+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:29.768229+0000","last_peered":"2024-09-16T04:57:29.768229+0000","last_clean":"2024-09-16T04:57:29.768229+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:29.768229+0000","last_undegraded":"2024-09-16T04:57:29.768229+0000","last_fullsized":"2024-09-16T04:57:29.768229+0000","mapping_epoch":34,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9023488,"num_objects":2203,"num_object_clones":0,"num_object_copies":4406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14883,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'17870","reported_seq":18108,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.529236+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:29.529236+0000","last_peered":"2024-09-16T04:57:29.529236+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:29.529236+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9056256,"num_objects":2211,"num_object_clones":0,"num_object_copies":4422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14621,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1110}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'17991","reported_seq":18131,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.738780+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:29.738780+0000","last_peered":"2024-09-16T04:57:29.738780+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:29.738780+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9121792,"num_objects":2227,"num_object_clones":0,"num_object_copies":4454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14646,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1238}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17727","reported_seq":24582,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.489992+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:29.489992+0000","last_peered":"2024-09-16T04:57:29.489992+0000","last_clean":"2024-09-16T04:57:29.489992+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:29.489992+0000","last_undegraded":"2024-09-16T04:57:29.489992+0000","last_fullsized":"2024-09-16T04:57:29.489992+0000","mapping_epoch":50,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:32.853641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9225426,"num_objects":2254,"num_object_clones":0,"num_object_copies":4508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17699,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17782","reported_seq":24649,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.564123+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:29.564123+0000","last_peered":"2024-09-16T04:57:29.564123+0000","last_clean":"2024-09-16T04:57:29.564123+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:29.564123+0000","last_undegraded":"2024-09-16T04:57:29.564123+0000","last_fullsized":"2024-09-16T04:57:29.564123+0000","mapping_epoch":46,"log_start":"47'7700","ondisk_log_start":"47'7700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8994816,"num_objects":2196,"num_object_clones":0,"num_object_copies":4392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14505,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17691","reported_seq":17797,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.485848+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:29.485848+0000","last_peered":"2024-09-16T04:57:29.485848+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:29.485848+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'7600","ondisk_log_start":"47'7600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8950377,"num_objects":2186,"num_object_clones":0,"num_object_copies":4372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17690,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1221}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18242","reported_seq":18355,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.529357+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:29.529357+0000","last_peered":"2024-09-16T04:57:29.529357+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:29.529357+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9381691,"num_objects":2293,"num_object_clones":0,"num_object_copies":4586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18239,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1221}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'17963","reported_seq":24767,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:29.165320+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:29.165320+0000","last_peered":"2024-09-16T04:57:29.165320+0000","last_clean":"2024-09-16T04:57:29.165320+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:29.165320+0000","last_undegraded":"2024-09-16T04:57:29.165320+0000","last_fullsized":"2024-09-16T04:57:29.165320+0000","mapping_epoch":48,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9154560,"num_objects":2235,"num_object_clones":0,"num_object_copies":4470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14717,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18159","reported_seq":18309,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:29.710601+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:29.710601+0000","last_peered":"2024-09-16T04:57:29.710601+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:29.710601+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9245289,"num_objects":2258,"num_object_clones":0,"num_object_copies":4516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14928,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":145555703,"num_objects":35543,"num_object_clones":0,"num_object_copies":71086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251087,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10830,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":458297344,"data_stored":458223099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569112,"kb_used_data":172288,"kb_used_omap":4,"kb_used_meta":396795,"kb_avail":93802728,"statfs":{"total":96636764160,"available":96053993472,"internally_reserved":0,"allocated":176422912,"data_stored":176129265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":406318761},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593823,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533460,"kb_used_data":108864,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93838380,"statfs":{"total":96636764160,"available":96090501120,"internally_reserved":0,"allocated":111476736,"data_stored":111127425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":434761390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562960,"kb_used_data":170368,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93808880,"statfs":{"total":96636764160,"available":96060293120,"internally_reserved":0,"allocated":174456832,"data_stored":174167257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":173572096,"data_stored":173568617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109707264,"data_stored":109644049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":175017984,"data_stored":175010433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:37.435 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:38.214 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:38.215 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:38.288 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-09-16T04:57:36.488677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":136057098,"num_objects":33226,"num_object_clones":0,"num_object_copies":66452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29829,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253408,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10914,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160731,"ondisk_log_size":160731,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1663508,"kb_used_data":432868,"kb_used_omap":12,"kb_used_meta":1230579,"kb_avail":281452012,"statfs":{"total":289910292480,"available":288206860288,"internally_reserved":0,"allocated":443256832,"data_stored":442324299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1260112897},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,4,10,12,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28295168,"num_objects":-6908,"num_object_clones":0,"num_object_copies":-13816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-245,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6908,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002520"},"pg_stats":[{"pgid":"2.f","version":"51'17687","reported_seq":19644,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.705679+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:34.705679+0000","last_peered":"2024-09-16T04:57:34.705679+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:34.705679+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'7600","ondisk_log_start":"47'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8400896,"num_objects":2051,"num_object_clones":0,"num_object_copies":4102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17687,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":934,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-357}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17952","reported_seq":18077,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.587993+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:34.587993+0000","last_peered":"2024-09-16T04:57:34.587993+0000","last_clean":"2024-09-16T04:57:34.587993+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:34.587993+0000","last_undegraded":"2024-09-16T04:57:34.587993+0000","last_fullsized":"2024-09-16T04:57:34.587993+0000","mapping_epoch":35,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8364032,"num_objects":2042,"num_object_clones":0,"num_object_copies":4084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14921,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17818","reported_seq":17967,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.526056+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:34.526056+0000","last_peered":"2024-09-16T04:57:34.526056+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:34.526056+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8351768,"num_objects":2040,"num_object_clones":0,"num_object_copies":4080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14557,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18201","reported_seq":18281,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.541369+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:34.541369+0000","last_peered":"2024-09-16T04:57:34.541369+0000","last_clean":"2024-09-16T04:57:34.541369+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:34.541369+0000","last_undegraded":"2024-09-16T04:57:34.541369+0000","last_fullsized":"2024-09-16T04:57:34.541369+0000","mapping_epoch":35,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8777728,"num_objects":2143,"num_object_clones":0,"num_object_copies":4286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18201,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18065","reported_seq":18294,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.625044+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:34.625044+0000","last_peered":"2024-09-16T04:57:34.625044+0000","last_clean":"2024-09-16T04:57:34.625044+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:34.625044+0000","last_undegraded":"2024-09-16T04:57:34.625044+0000","last_fullsized":"2024-09-16T04:57:34.625044+0000","mapping_epoch":34,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8515584,"num_objects":2079,"num_object_clones":0,"num_object_copies":4158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15097,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18200","reported_seq":18341,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.634349+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:34.634349+0000","last_peered":"2024-09-16T04:57:34.634349+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:34.634349+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8654848,"num_objects":2113,"num_object_clones":0,"num_object_copies":4226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15193,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17844","reported_seq":18003,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.732812+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:34.732812+0000","last_peered":"2024-09-16T04:57:34.732812+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:34.732812+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8257536,"num_objects":2016,"num_object_clones":0,"num_object_copies":4032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14499,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17948","reported_seq":18173,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.755967+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:34.755967+0000","last_peered":"2024-09-16T04:57:34.755967+0000","last_clean":"2024-09-16T04:57:34.755967+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:34.755967+0000","last_undegraded":"2024-09-16T04:57:34.755967+0000","last_fullsized":"2024-09-16T04:57:34.755967+0000","mapping_epoch":34,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8470528,"num_objects":2068,"num_object_clones":0,"num_object_copies":4136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15018,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18029","reported_seq":18267,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.535993+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:34.535993+0000","last_peered":"2024-09-16T04:57:34.535993+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:34.535993+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8404992,"num_objects":2052,"num_object_clones":0,"num_object_copies":4104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14780,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1269}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18150","reported_seq":18289,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.736103+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:34.736103+0000","last_peered":"2024-09-16T04:57:34.736103+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:34.736103+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8470528,"num_objects":2068,"num_object_clones":0,"num_object_copies":4136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14805,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1397}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17859","reported_seq":24714,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.564832+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:34.564832+0000","last_peered":"2024-09-16T04:57:34.564832+0000","last_clean":"2024-09-16T04:57:34.564832+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:34.564832+0000","last_undegraded":"2024-09-16T04:57:34.564832+0000","last_fullsized":"2024-09-16T04:57:34.564832+0000","mapping_epoch":50,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:38.846852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8684754,"num_objects":2122,"num_object_clones":0,"num_object_copies":4244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17831,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17923","reported_seq":24789,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.594371+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:34.594371+0000","last_peered":"2024-09-16T04:57:34.594371+0000","last_clean":"2024-09-16T04:57:34.594371+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:34.594371+0000","last_undegraded":"2024-09-16T04:57:34.594371+0000","last_fullsized":"2024-09-16T04:57:34.594371+0000","mapping_epoch":46,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8417280,"num_objects":2055,"num_object_clones":0,"num_object_copies":4110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14646,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17839","reported_seq":17947,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.429736+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:34.429736+0000","last_peered":"2024-09-16T04:57:34.429736+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:34.429736+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8344169,"num_objects":2038,"num_object_clones":0,"num_object_copies":4076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17838,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1369}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18401","reported_seq":18514,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.560860+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:34.560860+0000","last_peered":"2024-09-16T04:57:34.560860+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:34.560860+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8730427,"num_objects":2134,"num_object_clones":0,"num_object_copies":4268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18398,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1380}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18120","reported_seq":24924,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.192624+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:34.192624+0000","last_peered":"2024-09-16T04:57:34.192624+0000","last_clean":"2024-09-16T04:57:34.192624+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:34.192624+0000","last_undegraded":"2024-09-16T04:57:34.192624+0000","last_fullsized":"2024-09-16T04:57:34.192624+0000","mapping_epoch":48,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8511488,"num_objects":2078,"num_object_clones":0,"num_object_copies":4156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14874,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18292","reported_seq":18442,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.750224+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:34.750224+0000","last_peered":"2024-09-16T04:57:34.750224+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:34.750224+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8700521,"num_objects":2125,"num_object_clones":0,"num_object_copies":4250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15061,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":136057079,"num_objects":33224,"num_object_clones":0,"num_object_copies":66448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29829,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33224,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253406,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10913,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":439332864,"data_stored":439258619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160728,"ondisk_log_size":160728,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567416,"kb_used_data":166440,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93804424,"statfs":{"total":96636764160,"available":96055730176,"internally_reserved":0,"allocated":170434560,"data_stored":170140913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":410578601},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593824,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534796,"kb_used_data":101884,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93837044,"statfs":{"total":96636764160,"available":96089133056,"internally_reserved":0,"allocated":104329216,"data_stored":103979905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443281070},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561296,"kb_used_data":164544,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93810544,"statfs":{"total":96636764160,"available":96061997056,"internally_reserved":0,"allocated":168493056,"data_stored":168203481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":167739392,"data_stored":167735913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102518784,"data_stored":102455569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":169074688,"data_stored":169067137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:38.290 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:39.194 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:39.195 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:39.237 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-16T04:57:38.491419+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":136057098,"num_objects":33226,"num_object_clones":0,"num_object_copies":66452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29829,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253408,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10914,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160731,"ondisk_log_size":160731,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1663508,"kb_used_data":432868,"kb_used_omap":12,"kb_used_meta":1230579,"kb_avail":281452012,"statfs":{"total":289910292480,"available":288206860288,"internally_reserved":0,"allocated":443256832,"data_stored":442324299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1260112897},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,4,10,12,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19189760,"num_objects":-4685,"num_object_clones":0,"num_object_copies":-9370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4685,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004815"},"pg_stats":[{"pgid":"2.f","version":"51'17687","reported_seq":19644,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.705679+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:34.705679+0000","last_peered":"2024-09-16T04:57:34.705679+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:34.705679+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'7600","ondisk_log_start":"47'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8400896,"num_objects":2051,"num_object_clones":0,"num_object_copies":4102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17687,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":934,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-357}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'17952","reported_seq":18077,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.587993+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:34.587993+0000","last_peered":"2024-09-16T04:57:34.587993+0000","last_clean":"2024-09-16T04:57:34.587993+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:34.587993+0000","last_undegraded":"2024-09-16T04:57:34.587993+0000","last_fullsized":"2024-09-16T04:57:34.587993+0000","mapping_epoch":35,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8364032,"num_objects":2042,"num_object_clones":0,"num_object_copies":4084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14921,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17818","reported_seq":17967,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.526056+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:34.526056+0000","last_peered":"2024-09-16T04:57:34.526056+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:34.526056+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8351768,"num_objects":2040,"num_object_clones":0,"num_object_copies":4080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14557,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18201","reported_seq":18281,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.541369+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:34.541369+0000","last_peered":"2024-09-16T04:57:34.541369+0000","last_clean":"2024-09-16T04:57:34.541369+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:34.541369+0000","last_undegraded":"2024-09-16T04:57:34.541369+0000","last_fullsized":"2024-09-16T04:57:34.541369+0000","mapping_epoch":35,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8777728,"num_objects":2143,"num_object_clones":0,"num_object_copies":4286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18201,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18065","reported_seq":18294,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.625044+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:34.625044+0000","last_peered":"2024-09-16T04:57:34.625044+0000","last_clean":"2024-09-16T04:57:34.625044+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:34.625044+0000","last_undegraded":"2024-09-16T04:57:34.625044+0000","last_fullsized":"2024-09-16T04:57:34.625044+0000","mapping_epoch":34,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8515584,"num_objects":2079,"num_object_clones":0,"num_object_copies":4158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15097,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18200","reported_seq":18341,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.634349+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:34.634349+0000","last_peered":"2024-09-16T04:57:34.634349+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:34.634349+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8654848,"num_objects":2113,"num_object_clones":0,"num_object_copies":4226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15193,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'17844","reported_seq":18003,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.732812+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:34.732812+0000","last_peered":"2024-09-16T04:57:34.732812+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:34.732812+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8257536,"num_objects":2016,"num_object_clones":0,"num_object_copies":4032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14499,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'17948","reported_seq":18173,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.755967+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:34.755967+0000","last_peered":"2024-09-16T04:57:34.755967+0000","last_clean":"2024-09-16T04:57:34.755967+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:34.755967+0000","last_undegraded":"2024-09-16T04:57:34.755967+0000","last_fullsized":"2024-09-16T04:57:34.755967+0000","mapping_epoch":34,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8470528,"num_objects":2068,"num_object_clones":0,"num_object_copies":4136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15018,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710011+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710011+0000","last_peered":"2024-09-16T04:56:42.710011+0000","last_clean":"2024-09-16T04:56:42.710011+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710011+0000","last_undegraded":"2024-09-16T04:56:42.710011+0000","last_fullsized":"2024-09-16T04:56:42.710011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18029","reported_seq":18267,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.535993+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:34.535993+0000","last_peered":"2024-09-16T04:57:34.535993+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:34.535993+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8404992,"num_objects":2052,"num_object_clones":0,"num_object_copies":4104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14780,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1269}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18150","reported_seq":18289,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.736103+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:34.736103+0000","last_peered":"2024-09-16T04:57:34.736103+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:34.736103+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8470528,"num_objects":2068,"num_object_clones":0,"num_object_copies":4136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14805,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1397}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'17859","reported_seq":24714,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.564832+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:34.564832+0000","last_peered":"2024-09-16T04:57:34.564832+0000","last_clean":"2024-09-16T04:57:34.564832+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:34.564832+0000","last_undegraded":"2024-09-16T04:57:34.564832+0000","last_fullsized":"2024-09-16T04:57:34.564832+0000","mapping_epoch":50,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:38.846852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8684754,"num_objects":2122,"num_object_clones":0,"num_object_copies":4244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17831,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'17923","reported_seq":24789,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.594371+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:34.594371+0000","last_peered":"2024-09-16T04:57:34.594371+0000","last_clean":"2024-09-16T04:57:34.594371+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:34.594371+0000","last_undegraded":"2024-09-16T04:57:34.594371+0000","last_fullsized":"2024-09-16T04:57:34.594371+0000","mapping_epoch":46,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8417280,"num_objects":2055,"num_object_clones":0,"num_object_copies":4110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14646,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17839","reported_seq":17947,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.429736+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:34.429736+0000","last_peered":"2024-09-16T04:57:34.429736+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:34.429736+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8344169,"num_objects":2038,"num_object_clones":0,"num_object_copies":4076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17838,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1369}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18401","reported_seq":18514,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.560860+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:34.560860+0000","last_peered":"2024-09-16T04:57:34.560860+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:34.560860+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8730427,"num_objects":2134,"num_object_clones":0,"num_object_copies":4268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18398,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1380}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18120","reported_seq":24924,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:34.192624+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:34.192624+0000","last_peered":"2024-09-16T04:57:34.192624+0000","last_clean":"2024-09-16T04:57:34.192624+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:34.192624+0000","last_undegraded":"2024-09-16T04:57:34.192624+0000","last_fullsized":"2024-09-16T04:57:34.192624+0000","mapping_epoch":48,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8511488,"num_objects":2078,"num_object_clones":0,"num_object_copies":4156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14874,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18292","reported_seq":18442,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:34.750224+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:34.750224+0000","last_peered":"2024-09-16T04:57:34.750224+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:34.750224+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8700521,"num_objects":2125,"num_object_clones":0,"num_object_copies":4250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15061,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":136057079,"num_objects":33224,"num_object_clones":0,"num_object_copies":66448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29829,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33224,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253406,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10913,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":439332864,"data_stored":439258619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160728,"ondisk_log_size":160728,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567416,"kb_used_data":166440,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93804424,"statfs":{"total":96636764160,"available":96055730176,"internally_reserved":0,"allocated":170434560,"data_stored":170140913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":410578601},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593824,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534796,"kb_used_data":101884,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93837044,"statfs":{"total":96636764160,"available":96089133056,"internally_reserved":0,"allocated":104329216,"data_stored":103979905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443281070},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":561296,"kb_used_data":164544,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93810544,"statfs":{"total":96636764160,"available":96061997056,"internally_reserved":0,"allocated":168493056,"data_stored":168203481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":167739392,"data_stored":167735913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102518784,"data_stored":102455569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":169074688,"data_stored":169067137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:39.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:57:39.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'17687", 'reported_seq': 19644, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:34.705679+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:57:34.705679+0000', 'last_peered': '2024-09-16T04:57:34.705679+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:57:34.705679+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'7600", 'ondisk_log_start': "47'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8400896, 'num_objects': 2051, 'num_object_clones': 0, 'num_object_copies': 4102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17687, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 934, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -357}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:39.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:57:39.238 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'17818", 'reported_seq': 17967, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:34.526056+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:57:34.526056+0000', 'last_peered': '2024-09-16T04:57:34.526056+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:57:34.526056+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'7800", 'ondisk_log_start': "47'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8351768, 'num_objects': 2040, 'num_object_clones': 0, 'num_object_copies': 4080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2040, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14557, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1279}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:39.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:57:39.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'18200", 'reported_seq': 18341, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:34.634349+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:57:34.634349+0000', 'last_peered': '2024-09-16T04:57:34.634349+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:57:34.634349+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'8100", 'ondisk_log_start': "47'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8654848, 'num_objects': 2113, 'num_object_clones': 0, 'num_object_copies': 4226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15193, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1343}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:39.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:57:39.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'17844", 'reported_seq': 18003, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:34.732812+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:57:34.732812+0000', 'last_peered': '2024-09-16T04:57:34.732812+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:57:34.732812+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'7800", 'ondisk_log_start': "47'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8257536, 'num_objects': 2016, 'num_object_clones': 0, 'num_object_copies': 4032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2016, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14499, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1383}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:39.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:57:39.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'18029", 'reported_seq': 18267, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:34.535993+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:57:34.535993+0000', 'last_peered': '2024-09-16T04:57:34.535993+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:57:34.535993+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'8000", 'ondisk_log_start': "47'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8404992, 'num_objects': 2052, 'num_object_clones': 0, 'num_object_copies': 4104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2052, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14780, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1269}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:39.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:57:39.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'18150", 'reported_seq': 18289, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:34.736103+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:57:34.736103+0000', 'last_peered': '2024-09-16T04:57:34.736103+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:57:34.736103+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'8100", 'ondisk_log_start': "47'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8470528, 'num_objects': 2068, 'num_object_clones': 0, 'num_object_copies': 4136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2068, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14805, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1397}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:39.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:57:39.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'17839", 'reported_seq': 17947, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:34.429736+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:57:34.429736+0000', 'last_peered': '2024-09-16T04:57:34.429736+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:57:34.429736+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'7800", 'ondisk_log_start': "47'7800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8344169, 'num_objects': 2038, 'num_object_clones': 0, 'num_object_copies': 4076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2038, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17838, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1369}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:39.239 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:57:39.240 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'18401", 'reported_seq': 18514, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:34.560860+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:57:34.560860+0000', 'last_peered': '2024-09-16T04:57:34.560860+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:57:34.560860+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'8400", 'ondisk_log_start': "47'8400", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8730427, 'num_objects': 2134, 'num_object_clones': 0, 'num_object_copies': 4268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2134, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18398, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1380}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:39.240 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:57:39.240 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'18292", 'reported_seq': 18442, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:34.750224+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:57:34.750224+0000', 'last_peered': '2024-09-16T04:57:34.750224+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:57:34.750224+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'8200", 'ondisk_log_start': "47'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8700521, 'num_objects': 2125, 'num_object_clones': 0, 'num_object_copies': 4250, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2125, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15061, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:39.240 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:57:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:40.285 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29829/66452 objects degraded (44.888%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":608,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":33226,"data_bytes":136057098,"bytes_used":1703432192,"bytes_avail":288206860288,"bytes_total":289910292480,"degraded_objects":29829,"degraded_total":66452,"degraded_ratio":0.44888039487148618,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":160,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":390},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:57:40.345 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:57:40.346 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:41.166 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:41.168 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:41.258 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-16T04:57:40.494497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":126357770,"num_objects":30858,"num_object_clones":0,"num_object_copies":61716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30858,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":255776,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11012,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160999,"ondisk_log_size":160999,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1626644,"kb_used_data":412608,"kb_used_omap":12,"kb_used_meta":1213939,"kb_avail":281488876,"statfs":{"total":289910292480,"available":288244609024,"internally_reserved":0,"allocated":422510592,"data_stored":421578059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1243073537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,5,3,8,12,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28889088,"num_objects":-7053,"num_object_clones":0,"num_object_copies":-14106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-258,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7053,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007543"},"pg_stats":[{"pgid":"2.f","version":"51'17841","reported_seq":19994,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.780987+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:39.780987+0000","last_peered":"2024-09-16T04:57:39.780987+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:39.780987+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7770112,"num_objects":1897,"num_object_clones":0,"num_object_copies":3794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17841,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1032,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-413}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18096","reported_seq":18222,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.401363+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:39.401363+0000","last_peered":"2024-09-16T04:57:39.401363+0000","last_clean":"2024-09-16T04:57:39.401363+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:39.401363+0000","last_undegraded":"2024-09-16T04:57:39.401363+0000","last_fullsized":"2024-09-16T04:57:39.401363+0000","mapping_epoch":35,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7774208,"num_objects":1898,"num_object_clones":0,"num_object_copies":3796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15065,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17950","reported_seq":18099,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.549956+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:39.549956+0000","last_peered":"2024-09-16T04:57:39.549956+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:39.549956+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7811096,"num_objects":1908,"num_object_clones":0,"num_object_copies":3816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1908,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14689,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1411}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18349","reported_seq":18429,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.552447+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:39.552447+0000","last_peered":"2024-09-16T04:57:39.552447+0000","last_clean":"2024-09-16T04:57:39.552447+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:39.552447+0000","last_undegraded":"2024-09-16T04:57:39.552447+0000","last_fullsized":"2024-09-16T04:57:39.552447+0000","mapping_epoch":35,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8171520,"num_objects":1995,"num_object_clones":0,"num_object_copies":3990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18349,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18192","reported_seq":18421,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.635809+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:39.635809+0000","last_peered":"2024-09-16T04:57:39.635809+0000","last_clean":"2024-09-16T04:57:39.635809+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:39.635809+0000","last_undegraded":"2024-09-16T04:57:39.635809+0000","last_fullsized":"2024-09-16T04:57:39.635809+0000","mapping_epoch":34,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7995392,"num_objects":1952,"num_object_clones":0,"num_object_copies":3904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15224,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18364","reported_seq":18507,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.748418+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:39.748418+0000","last_peered":"2024-09-16T04:57:39.748418+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:39.748418+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7983104,"num_objects":1949,"num_object_clones":0,"num_object_copies":3898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15357,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18010","reported_seq":18169,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.718851+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:39.718851+0000","last_peered":"2024-09-16T04:57:39.718851+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:39.718851+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7577600,"num_objects":1850,"num_object_clones":0,"num_object_copies":3700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14665,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1549}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18086","reported_seq":18312,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.599627+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:39.599627+0000","last_peered":"2024-09-16T04:57:39.599627+0000","last_clean":"2024-09-16T04:57:39.599627+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:39.599627+0000","last_undegraded":"2024-09-16T04:57:39.599627+0000","last_fullsized":"2024-09-16T04:57:39.599627+0000","mapping_epoch":34,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7905280,"num_objects":1930,"num_object_clones":0,"num_object_copies":3860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15156,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18174","reported_seq":18411,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.596934+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:39.596934+0000","last_peered":"2024-09-16T04:57:39.596934+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:39.596934+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7811072,"num_objects":1907,"num_object_clones":0,"num_object_copies":3814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14925,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1414}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18299","reported_seq":18437,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.706293+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:39.706293+0000","last_peered":"2024-09-16T04:57:39.706293+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:39.706293+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7860224,"num_objects":1919,"num_object_clones":0,"num_object_copies":3838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14954,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1546}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18005","reported_seq":24860,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.602355+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:39.602355+0000","last_peered":"2024-09-16T04:57:39.602355+0000","last_clean":"2024-09-16T04:57:39.602355+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:39.602355+0000","last_undegraded":"2024-09-16T04:57:39.602355+0000","last_fullsized":"2024-09-16T04:57:39.602355+0000","mapping_epoch":50,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8086738,"num_objects":1976,"num_object_clones":0,"num_object_copies":3952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17977,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18079","reported_seq":24946,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.558315+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:39.558315+0000","last_peered":"2024-09-16T04:57:39.558315+0000","last_clean":"2024-09-16T04:57:39.558315+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:39.558315+0000","last_undegraded":"2024-09-16T04:57:39.558315+0000","last_fullsized":"2024-09-16T04:57:39.558315+0000","mapping_epoch":46,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7778304,"num_objects":1899,"num_object_clones":0,"num_object_copies":3798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14802,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17984","reported_seq":18091,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.495858+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:39.495858+0000","last_peered":"2024-09-16T04:57:39.495858+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:39.495858+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7750249,"num_objects":1893,"num_object_clones":0,"num_object_copies":3786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17983,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18533","reported_seq":18646,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.542642+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:39.542642+0000","last_peered":"2024-09-16T04:57:39.542642+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:39.542642+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8189755,"num_objects":2002,"num_object_clones":0,"num_object_copies":4004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18530,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1512}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18286","reported_seq":25090,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.264410+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:39.264410+0000","last_peered":"2024-09-16T04:57:39.264410+0000","last_clean":"2024-09-16T04:57:39.264410+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:39.264410+0000","last_undegraded":"2024-09-16T04:57:39.264410+0000","last_fullsized":"2024-09-16T04:57:39.264410+0000","mapping_epoch":48,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7831552,"num_objects":1912,"num_object_clones":0,"num_object_copies":3824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15040,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18448","reported_seq":18598,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.782049+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:39.782049+0000","last_peered":"2024-09-16T04:57:39.782049+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:39.782049+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8061545,"num_objects":1969,"num_object_clones":0,"num_object_copies":3938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15217,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":126357751,"num_objects":30856,"num_object_clones":0,"num_object_copies":61712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":255774,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11011,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":419938304,"data_stored":419864059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160996,"ondisk_log_size":160996,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549424,"kb_used_data":160904,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93822416,"statfs":{"total":96636764160,"available":96074153984,"internally_reserved":0,"allocated":164765696,"data_stored":164472049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":397799081},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,1,4,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593825,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517928,"kb_used_data":93332,"kb_used_omap":4,"kb_used_meta":424571,"kb_avail":93853912,"statfs":{"total":96636764160,"available":96106405888,"internally_reserved":0,"allocated":95571968,"data_stored":95222657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":434761390},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559292,"kb_used_data":158372,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93812548,"statfs":{"total":96636764160,"available":96064049152,"internally_reserved":0,"allocated":162172928,"data_stored":161883353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":161566720,"data_stored":161563241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95092736,"data_stored":95029521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163278848,"data_stored":163271297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:44.261 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:45.161 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:45.162 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:45.221 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-16T04:57:44.495429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":125796618,"num_objects":30721,"num_object_clones":0,"num_object_copies":61442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30721,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":255913,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11012,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160936,"ondisk_log_size":160936,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1627852,"kb_used_data":405500,"kb_used_omap":12,"kb_used_meta":1222259,"kb_avail":281487668,"statfs":{"total":289910292480,"available":288243372032,"internally_reserved":0,"allocated":415232000,"data_stored":414299467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1251593217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,4,6,11,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19759104,"num_objects":-4824,"num_object_clones":0,"num_object_copies":-9648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4824,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007644"},"pg_stats":[{"pgid":"2.f","version":"51'17841","reported_seq":19994,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.780987+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:39.780987+0000","last_peered":"2024-09-16T04:57:39.780987+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:39.780987+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7770112,"num_objects":1897,"num_object_clones":0,"num_object_copies":3794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17841,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1032,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-413}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18096","reported_seq":18222,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.401363+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:39.401363+0000","last_peered":"2024-09-16T04:57:39.401363+0000","last_clean":"2024-09-16T04:57:39.401363+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:39.401363+0000","last_undegraded":"2024-09-16T04:57:39.401363+0000","last_fullsized":"2024-09-16T04:57:39.401363+0000","mapping_epoch":35,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7774208,"num_objects":1898,"num_object_clones":0,"num_object_copies":3796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15065,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17950","reported_seq":18099,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.549956+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:39.549956+0000","last_peered":"2024-09-16T04:57:39.549956+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:39.549956+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7811096,"num_objects":1908,"num_object_clones":0,"num_object_copies":3816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1908,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14689,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1411}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18349","reported_seq":18429,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.552447+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:39.552447+0000","last_peered":"2024-09-16T04:57:39.552447+0000","last_clean":"2024-09-16T04:57:39.552447+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:39.552447+0000","last_undegraded":"2024-09-16T04:57:39.552447+0000","last_fullsized":"2024-09-16T04:57:39.552447+0000","mapping_epoch":35,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8171520,"num_objects":1995,"num_object_clones":0,"num_object_copies":3990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18349,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18192","reported_seq":18421,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.635809+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:39.635809+0000","last_peered":"2024-09-16T04:57:39.635809+0000","last_clean":"2024-09-16T04:57:39.635809+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:39.635809+0000","last_undegraded":"2024-09-16T04:57:39.635809+0000","last_fullsized":"2024-09-16T04:57:39.635809+0000","mapping_epoch":34,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7995392,"num_objects":1952,"num_object_clones":0,"num_object_copies":3904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15224,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18364","reported_seq":18507,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.748418+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:39.748418+0000","last_peered":"2024-09-16T04:57:39.748418+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:39.748418+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7983104,"num_objects":1949,"num_object_clones":0,"num_object_copies":3898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15357,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18010","reported_seq":18169,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.718851+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:39.718851+0000","last_peered":"2024-09-16T04:57:39.718851+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:39.718851+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7577600,"num_objects":1850,"num_object_clones":0,"num_object_copies":3700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14665,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1549}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18086","reported_seq":18312,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.599627+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:39.599627+0000","last_peered":"2024-09-16T04:57:39.599627+0000","last_clean":"2024-09-16T04:57:39.599627+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:39.599627+0000","last_undegraded":"2024-09-16T04:57:39.599627+0000","last_fullsized":"2024-09-16T04:57:39.599627+0000","mapping_epoch":34,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7905280,"num_objects":1930,"num_object_clones":0,"num_object_copies":3860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15156,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18174","reported_seq":18411,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.596934+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:39.596934+0000","last_peered":"2024-09-16T04:57:39.596934+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:39.596934+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7811072,"num_objects":1907,"num_object_clones":0,"num_object_copies":3814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14925,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1414}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18299","reported_seq":18437,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.706293+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:39.706293+0000","last_peered":"2024-09-16T04:57:39.706293+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:39.706293+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7860224,"num_objects":1919,"num_object_clones":0,"num_object_copies":3838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14954,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1546}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18005","reported_seq":24860,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.602355+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:39.602355+0000","last_peered":"2024-09-16T04:57:39.602355+0000","last_clean":"2024-09-16T04:57:39.602355+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:39.602355+0000","last_undegraded":"2024-09-16T04:57:39.602355+0000","last_fullsized":"2024-09-16T04:57:39.602355+0000","mapping_epoch":50,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8086738,"num_objects":1976,"num_object_clones":0,"num_object_copies":3952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17977,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18079","reported_seq":24946,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.558315+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:39.558315+0000","last_peered":"2024-09-16T04:57:39.558315+0000","last_clean":"2024-09-16T04:57:39.558315+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:39.558315+0000","last_undegraded":"2024-09-16T04:57:39.558315+0000","last_fullsized":"2024-09-16T04:57:39.558315+0000","mapping_epoch":46,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7778304,"num_objects":1899,"num_object_clones":0,"num_object_copies":3798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14802,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17984","reported_seq":18091,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.495858+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:39.495858+0000","last_peered":"2024-09-16T04:57:39.495858+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:39.495858+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7750249,"num_objects":1893,"num_object_clones":0,"num_object_copies":3786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17983,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18533","reported_seq":18646,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.542642+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:39.542642+0000","last_peered":"2024-09-16T04:57:39.542642+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:39.542642+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8189755,"num_objects":2002,"num_object_clones":0,"num_object_copies":4004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18530,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1512}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18423","reported_seq":25227,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:44.236917+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:44.236917+0000","last_peered":"2024-09-16T04:57:44.236917+0000","last_clean":"2024-09-16T04:57:44.236917+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:44.236917+0000","last_undegraded":"2024-09-16T04:57:44.236917+0000","last_fullsized":"2024-09-16T04:57:44.236917+0000","mapping_epoch":48,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7270400,"num_objects":1775,"num_object_clones":0,"num_object_copies":3550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15177,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18448","reported_seq":18598,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.782049+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:39.782049+0000","last_peered":"2024-09-16T04:57:39.782049+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:39.782049+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8061545,"num_objects":1969,"num_object_clones":0,"num_object_copies":3938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15217,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":125796599,"num_objects":30719,"num_object_clones":0,"num_object_copies":61438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30719,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":255911,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11011,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":412606464,"data_stored":412532219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549424,"kb_used_data":160904,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93822416,"statfs":{"total":96636764160,"available":96074153984,"internally_reserved":0,"allocated":164765696,"data_stored":164472049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":397799081},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,1,4,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593826,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519136,"kb_used_data":86224,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93852704,"statfs":{"total":96636764160,"available":96105168896,"internally_reserved":0,"allocated":88293376,"data_stored":87944065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443281070},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559292,"kb_used_data":158372,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93812548,"statfs":{"total":96636764160,"available":96064049152,"internally_reserved":0,"allocated":162172928,"data_stored":161883353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":161566720,"data_stored":161563241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":87760896,"data_stored":87697681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163278848,"data_stored":163271297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:45.222 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:46.136 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:46.138 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:46.200 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-16T04:57:44.495429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":125796618,"num_objects":30721,"num_object_clones":0,"num_object_copies":61442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30721,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":255913,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11012,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160936,"ondisk_log_size":160936,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1627852,"kb_used_data":405500,"kb_used_omap":12,"kb_used_meta":1222259,"kb_avail":281487668,"statfs":{"total":289910292480,"available":288243372032,"internally_reserved":0,"allocated":415232000,"data_stored":414299467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1251593217},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,4,6,11,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19759104,"num_objects":-4824,"num_object_clones":0,"num_object_copies":-9648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4824,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007644"},"pg_stats":[{"pgid":"2.f","version":"51'17841","reported_seq":19994,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.780987+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:39.780987+0000","last_peered":"2024-09-16T04:57:39.780987+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:39.780987+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'7800","ondisk_log_start":"47'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7770112,"num_objects":1897,"num_object_clones":0,"num_object_copies":3794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17841,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1032,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-413}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18096","reported_seq":18222,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.401363+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:39.401363+0000","last_peered":"2024-09-16T04:57:39.401363+0000","last_clean":"2024-09-16T04:57:39.401363+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:39.401363+0000","last_undegraded":"2024-09-16T04:57:39.401363+0000","last_fullsized":"2024-09-16T04:57:39.401363+0000","mapping_epoch":35,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7774208,"num_objects":1898,"num_object_clones":0,"num_object_copies":3796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15065,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'17950","reported_seq":18099,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.549956+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:39.549956+0000","last_peered":"2024-09-16T04:57:39.549956+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:39.549956+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7811096,"num_objects":1908,"num_object_clones":0,"num_object_copies":3816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1908,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14689,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1411}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18349","reported_seq":18429,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.552447+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:39.552447+0000","last_peered":"2024-09-16T04:57:39.552447+0000","last_clean":"2024-09-16T04:57:39.552447+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:39.552447+0000","last_undegraded":"2024-09-16T04:57:39.552447+0000","last_fullsized":"2024-09-16T04:57:39.552447+0000","mapping_epoch":35,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8171520,"num_objects":1995,"num_object_clones":0,"num_object_copies":3990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18349,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18192","reported_seq":18421,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.635809+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:39.635809+0000","last_peered":"2024-09-16T04:57:39.635809+0000","last_clean":"2024-09-16T04:57:39.635809+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:39.635809+0000","last_undegraded":"2024-09-16T04:57:39.635809+0000","last_fullsized":"2024-09-16T04:57:39.635809+0000","mapping_epoch":34,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7995392,"num_objects":1952,"num_object_clones":0,"num_object_copies":3904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15224,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18364","reported_seq":18507,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.748418+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:39.748418+0000","last_peered":"2024-09-16T04:57:39.748418+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:39.748418+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7983104,"num_objects":1949,"num_object_clones":0,"num_object_copies":3898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15357,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18010","reported_seq":18169,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.718851+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:39.718851+0000","last_peered":"2024-09-16T04:57:39.718851+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:39.718851+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7577600,"num_objects":1850,"num_object_clones":0,"num_object_copies":3700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14665,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1549}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18086","reported_seq":18312,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.599627+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:39.599627+0000","last_peered":"2024-09-16T04:57:39.599627+0000","last_clean":"2024-09-16T04:57:39.599627+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:39.599627+0000","last_undegraded":"2024-09-16T04:57:39.599627+0000","last_fullsized":"2024-09-16T04:57:39.599627+0000","mapping_epoch":34,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7905280,"num_objects":1930,"num_object_clones":0,"num_object_copies":3860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15156,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18174","reported_seq":18411,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.596934+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:39.596934+0000","last_peered":"2024-09-16T04:57:39.596934+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:39.596934+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7811072,"num_objects":1907,"num_object_clones":0,"num_object_copies":3814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14925,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1414}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18299","reported_seq":18437,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.706293+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:39.706293+0000","last_peered":"2024-09-16T04:57:39.706293+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:39.706293+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7860224,"num_objects":1919,"num_object_clones":0,"num_object_copies":3838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14954,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1546}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18005","reported_seq":24860,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.602355+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:39.602355+0000","last_peered":"2024-09-16T04:57:39.602355+0000","last_clean":"2024-09-16T04:57:39.602355+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:39.602355+0000","last_undegraded":"2024-09-16T04:57:39.602355+0000","last_fullsized":"2024-09-16T04:57:39.602355+0000","mapping_epoch":50,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8086738,"num_objects":1976,"num_object_clones":0,"num_object_copies":3952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17977,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18079","reported_seq":24946,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:39.558315+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:39.558315+0000","last_peered":"2024-09-16T04:57:39.558315+0000","last_clean":"2024-09-16T04:57:39.558315+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:39.558315+0000","last_undegraded":"2024-09-16T04:57:39.558315+0000","last_fullsized":"2024-09-16T04:57:39.558315+0000","mapping_epoch":46,"log_start":"47'8000","ondisk_log_start":"47'8000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7778304,"num_objects":1899,"num_object_clones":0,"num_object_copies":3798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14802,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'17984","reported_seq":18091,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.495858+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:39.495858+0000","last_peered":"2024-09-16T04:57:39.495858+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:39.495858+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7750249,"num_objects":1893,"num_object_clones":0,"num_object_copies":3786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17983,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18533","reported_seq":18646,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.542642+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:39.542642+0000","last_peered":"2024-09-16T04:57:39.542642+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:39.542642+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8189755,"num_objects":2002,"num_object_clones":0,"num_object_copies":4004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18530,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1512}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18423","reported_seq":25227,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:44.236917+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:44.236917+0000","last_peered":"2024-09-16T04:57:44.236917+0000","last_clean":"2024-09-16T04:57:44.236917+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:44.236917+0000","last_undegraded":"2024-09-16T04:57:44.236917+0000","last_fullsized":"2024-09-16T04:57:44.236917+0000","mapping_epoch":48,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7270400,"num_objects":1775,"num_object_clones":0,"num_object_copies":3550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15177,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18448","reported_seq":18598,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:39.782049+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:39.782049+0000","last_peered":"2024-09-16T04:57:39.782049+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:39.782049+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8061545,"num_objects":1969,"num_object_clones":0,"num_object_copies":3938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15217,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":125796599,"num_objects":30719,"num_object_clones":0,"num_object_copies":61438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30719,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":255911,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11011,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":412606464,"data_stored":412532219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549424,"kb_used_data":160904,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93822416,"statfs":{"total":96636764160,"available":96074153984,"internally_reserved":0,"allocated":164765696,"data_stored":164472049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":397799081},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,1,4,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593826,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519136,"kb_used_data":86224,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93852704,"statfs":{"total":96636764160,"available":96105168896,"internally_reserved":0,"allocated":88293376,"data_stored":87944065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443281070},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559292,"kb_used_data":158372,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93812548,"statfs":{"total":96636764160,"available":96064049152,"internally_reserved":0,"allocated":162172928,"data_stored":161883353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":161566720,"data_stored":161563241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":87760896,"data_stored":87697681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163278848,"data_stored":163271297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:46.202 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:57:46.202 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'17841", 'reported_seq': 19994, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:39.780987+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:57:39.780987+0000', 'last_peered': '2024-09-16T04:57:39.780987+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:57:39.780987+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'7800", 'ondisk_log_start': "47'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7770112, 'num_objects': 1897, 'num_object_clones': 0, 'num_object_copies': 3794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2310, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1897, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17841, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1032, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -413}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:46.202 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:57:46.202 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'17950", 'reported_seq': 18099, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:39.549956+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:57:39.549956+0000', 'last_peered': '2024-09-16T04:57:39.549956+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:57:39.549956+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'7900", 'ondisk_log_start': "47'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7811096, 'num_objects': 1908, 'num_object_clones': 0, 'num_object_copies': 3816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1908, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14689, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1411}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:46.202 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:57:46.202 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'18364", 'reported_seq': 18507, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:39.748418+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:57:39.748418+0000', 'last_peered': '2024-09-16T04:57:39.748418+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:57:39.748418+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'8300", 'ondisk_log_start': "47'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7983104, 'num_objects': 1949, 'num_object_clones': 0, 'num_object_copies': 3898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1949, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15357, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1507}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:46.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:57:46.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'18010", 'reported_seq': 18169, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:39.718851+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:57:39.718851+0000', 'last_peered': '2024-09-16T04:57:39.718851+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:57:39.718851+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'8000", 'ondisk_log_start': "47'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7577600, 'num_objects': 1850, 'num_object_clones': 0, 'num_object_copies': 3700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1850, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14665, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1549}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:46.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:57:46.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'18174", 'reported_seq': 18411, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:39.596934+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:57:39.596934+0000', 'last_peered': '2024-09-16T04:57:39.596934+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:57:39.596934+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'8100", 'ondisk_log_start': "47'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7811072, 'num_objects': 1907, 'num_object_clones': 0, 'num_object_copies': 3814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1907, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14925, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1414}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:46.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:57:46.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'18299", 'reported_seq': 18437, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:39.706293+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:57:39.706293+0000', 'last_peered': '2024-09-16T04:57:39.706293+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:57:39.706293+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'8200", 'ondisk_log_start': "47'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7860224, 'num_objects': 1919, 'num_object_clones': 0, 'num_object_copies': 3838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1919, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14954, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1546}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:46.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:57:46.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'17984", 'reported_seq': 18091, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:39.495858+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:57:39.495858+0000', 'last_peered': '2024-09-16T04:57:39.495858+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:57:39.495858+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'7900", 'ondisk_log_start': "47'7900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7750249, 'num_objects': 1893, 'num_object_clones': 0, 'num_object_copies': 3786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17983, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:46.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:57:46.203 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'18533", 'reported_seq': 18646, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:39.542642+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:57:39.542642+0000', 'last_peered': '2024-09-16T04:57:39.542642+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:57:39.542642+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'8500", 'ondisk_log_start': "47'8500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8189755, 'num_objects': 2002, 'num_object_clones': 0, 'num_object_copies': 4004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2002, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18530, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1512}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:46.204 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:57:46.204 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'18448", 'reported_seq': 18598, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:39.782049+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:57:39.782049+0000', 'last_peered': '2024-09-16T04:57:39.782049+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:57:39.782049+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'8400", 'ondisk_log_start': "47'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8061545, 'num_objects': 1969, 'num_object_clones': 0, 'num_object_copies': 3938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1969, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15217, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1555}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:46.204 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:57:47.176 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:47.177 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29642/56948 objects degraded (52.051%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":614,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":28474,"data_bytes":116592906,"bytes_used":1650503680,"bytes_avail":288259788800,"bytes_total":289910292480,"degraded_objects":29642,"degraded_total":56948,"degraded_ratio":0.52050993889162045,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":575},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:57:47.240 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:57:47.240 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:48.024 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:48.084 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-09-16T04:57:46.496820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":116592906,"num_objects":28474,"num_object_clones":0,"num_object_copies":56948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258160,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11101,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160683,"ondisk_log_size":160683,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1611820,"kb_used_data":393656,"kb_used_omap":12,"kb_used_meta":1218099,"kb_avail":281503700,"statfs":{"total":289910292480,"available":288259788800,"internally_reserved":0,"allocated":403103744,"data_stored":402171211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1247333377},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,3,3,5,14,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28319744,"num_objects":-6914,"num_object_clones":0,"num_object_copies":-13828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6914,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008621"},"pg_stats":[{"pgid":"2.f","version":"51'17992","reported_seq":20323,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:44.746420+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:44.746420+0000","last_peered":"2024-09-16T04:57:44.746420+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:44.746420+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'7900","ondisk_log_start":"47'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7151616,"num_objects":1746,"num_object_clones":0,"num_object_copies":3492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17992,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1121,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-475}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18251","reported_seq":18377,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:44.563172+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:44.563172+0000","last_peered":"2024-09-16T04:57:44.563172+0000","last_clean":"2024-09-16T04:57:44.563172+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:44.563172+0000","last_undegraded":"2024-09-16T04:57:44.563172+0000","last_fullsized":"2024-09-16T04:57:44.563172+0000","mapping_epoch":35,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7139328,"num_objects":1743,"num_object_clones":0,"num_object_copies":3486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15220,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18108","reported_seq":18256,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:44.649675+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:44.649675+0000","last_peered":"2024-09-16T04:57:44.649675+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:44.649675+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7163928,"num_objects":1750,"num_object_clones":0,"num_object_copies":3500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1750,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14847,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1569}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18509","reported_seq":18589,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:44.630762+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:44.630762+0000","last_peered":"2024-09-16T04:57:44.630762+0000","last_clean":"2024-09-16T04:57:44.630762+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:44.630762+0000","last_undegraded":"2024-09-16T04:57:44.630762+0000","last_fullsized":"2024-09-16T04:57:44.630762+0000","mapping_epoch":35,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7516160,"num_objects":1835,"num_object_clones":0,"num_object_copies":3670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18509,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18336","reported_seq":18565,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:44.768431+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:44.768431+0000","last_peered":"2024-09-16T04:57:44.768431+0000","last_clean":"2024-09-16T04:57:44.768431+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:44.768431+0000","last_undegraded":"2024-09-16T04:57:44.768431+0000","last_fullsized":"2024-09-16T04:57:44.768431+0000","mapping_epoch":34,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7405568,"num_objects":1808,"num_object_clones":0,"num_object_copies":3616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15368,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18517","reported_seq":18660,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:44.776887+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:44.776887+0000","last_peered":"2024-09-16T04:57:44.776887+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:44.776887+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7356416,"num_objects":1796,"num_object_clones":0,"num_object_copies":3592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15510,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18159","reported_seq":18318,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:44.741993+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:44.741993+0000","last_peered":"2024-09-16T04:57:44.741993+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:44.741993+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6967296,"num_objects":1701,"num_object_clones":0,"num_object_copies":3402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14814,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1698}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18239","reported_seq":18465,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:44.705246+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:44.705246+0000","last_peered":"2024-09-16T04:57:44.705246+0000","last_clean":"2024-09-16T04:57:44.705246+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:44.705246+0000","last_undegraded":"2024-09-16T04:57:44.705246+0000","last_fullsized":"2024-09-16T04:57:44.705246+0000","mapping_epoch":34,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7278592,"num_objects":1777,"num_object_clones":0,"num_object_copies":3554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15309,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18345","reported_seq":18582,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:44.612367+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:44.612367+0000","last_peered":"2024-09-16T04:57:44.612367+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:44.612367+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7110656,"num_objects":1736,"num_object_clones":0,"num_object_copies":3472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15096,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1585}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18422","reported_seq":18559,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:44.662525+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:44.662525+0000","last_peered":"2024-09-16T04:57:44.662525+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:44.662525+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7356416,"num_objects":1796,"num_object_clones":0,"num_object_copies":3592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15077,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1669}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18154","reported_seq":25010,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:44.601007+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:44.601007+0000","last_peered":"2024-09-16T04:57:44.601007+0000","last_clean":"2024-09-16T04:57:44.601007+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:44.601007+0000","last_undegraded":"2024-09-16T04:57:44.601007+0000","last_fullsized":"2024-09-16T04:57:44.601007+0000","mapping_epoch":50,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:44.734595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7476434,"num_objects":1827,"num_object_clones":0,"num_object_copies":3654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18126,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18222","reported_seq":25089,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:44.548135+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:44.548135+0000","last_peered":"2024-09-16T04:57:44.548135+0000","last_clean":"2024-09-16T04:57:44.548135+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:44.548135+0000","last_undegraded":"2024-09-16T04:57:44.548135+0000","last_fullsized":"2024-09-16T04:57:44.548135+0000","mapping_epoch":46,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7192576,"num_objects":1756,"num_object_clones":0,"num_object_copies":3512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14945,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18133","reported_seq":18241,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:44.437011+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:44.437011+0000","last_peered":"2024-09-16T04:57:44.437011+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:44.437011+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7139945,"num_objects":1744,"num_object_clones":0,"num_object_copies":3488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18132,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1663}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18675","reported_seq":18788,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:44.630312+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:44.630312+0000","last_peered":"2024-09-16T04:57:44.630312+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:44.630312+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7608123,"num_objects":1860,"num_object_clones":0,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18672,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1654}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18423","reported_seq":25227,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:44.236917+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:44.236917+0000","last_peered":"2024-09-16T04:57:44.236917+0000","last_clean":"2024-09-16T04:57:44.236917+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:44.236917+0000","last_undegraded":"2024-09-16T04:57:44.236917+0000","last_fullsized":"2024-09-16T04:57:44.236917+0000","mapping_epoch":48,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7270400,"num_objects":1775,"num_object_clones":0,"num_object_copies":3550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15177,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18595","reported_seq":18745,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:44.734343+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:44.734343+0000","last_peered":"2024-09-16T04:57:44.734343+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:44.734343+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7459433,"num_objects":1822,"num_object_clones":0,"num_object_copies":3644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15364,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1702}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":116592887,"num_objects":28472,"num_object_clones":0,"num_object_copies":56944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28472,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":258158,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11100,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":400318464,"data_stored":400244219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160680,"ondisk_log_size":160680,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547636,"kb_used_data":154980,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93824204,"statfs":{"total":96636764160,"available":96075984896,"internally_reserved":0,"allocated":158699520,"data_stored":158405873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":402058921},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,0,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593826,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519136,"kb_used_data":86224,"kb_used_omap":4,"kb_used_meta":432891,"kb_avail":93852704,"statfs":{"total":96636764160,"available":96105168896,"internally_reserved":0,"allocated":88293376,"data_stored":87944065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":443281070},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545048,"kb_used_data":152452,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93826792,"statfs":{"total":96636764160,"available":96078635008,"internally_reserved":0,"allocated":156110848,"data_stored":155821273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":155525120,"data_stored":155521641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":87760896,"data_stored":87697681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":157032448,"data_stored":157024897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:51.086 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:51.939 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:51.940 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:51.996 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-16T04:57:50.503526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":107061514,"num_objects":26147,"num_object_clones":0,"num_object_copies":52294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26147,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260487,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11186,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161010,"ondisk_log_size":161010,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1608636,"kb_used_data":373828,"kb_used_omap":12,"kb_used_meta":1234739,"kb_avail":281506884,"statfs":{"total":289910292480,"available":288263049216,"internally_reserved":0,"allocated":382799872,"data_stored":381867339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1264372737},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,3,7,8,10,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28995584,"num_objects":-7079,"num_object_clones":0,"num_object_copies":-14158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-272,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7079,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012107"},"pg_stats":[{"pgid":"2.f","version":"51'18140","reported_seq":20641,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.798458+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:49.798458+0000","last_peered":"2024-09-16T04:57:49.798458+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:49.798458+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6545408,"num_objects":1598,"num_object_clones":0,"num_object_copies":3196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1206,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-538}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18399","reported_seq":18526,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.621442+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:49.621442+0000","last_peered":"2024-09-16T04:57:49.621442+0000","last_clean":"2024-09-16T04:57:49.621442+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:49.621442+0000","last_undegraded":"2024-09-16T04:57:49.621442+0000","last_fullsized":"2024-09-16T04:57:49.621442+0000","mapping_epoch":35,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6533120,"num_objects":1595,"num_object_clones":0,"num_object_copies":3190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15368,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18279","reported_seq":18426,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.647081+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:49.647081+0000","last_peered":"2024-09-16T04:57:49.647081+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:49.647081+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6463512,"num_objects":1579,"num_object_clones":0,"num_object_copies":3158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15018,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1740}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18652","reported_seq":18733,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.614484+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:49.614484+0000","last_peered":"2024-09-16T04:57:49.614484+0000","last_clean":"2024-09-16T04:57:49.614484+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:49.614484+0000","last_undegraded":"2024-09-16T04:57:49.614484+0000","last_fullsized":"2024-09-16T04:57:49.614484+0000","mapping_epoch":35,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6930432,"num_objects":1692,"num_object_clones":0,"num_object_copies":3384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18652,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18491","reported_seq":18720,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.798920+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:49.798920+0000","last_peered":"2024-09-16T04:57:49.798920+0000","last_clean":"2024-09-16T04:57:49.798920+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:49.798920+0000","last_undegraded":"2024-09-16T04:57:49.798920+0000","last_fullsized":"2024-09-16T04:57:49.798920+0000","mapping_epoch":34,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6770688,"num_objects":1653,"num_object_clones":0,"num_object_copies":3306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15523,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18651","reported_seq":18794,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.793426+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:49.793426+0000","last_peered":"2024-09-16T04:57:49.793426+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:49.793426+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6807552,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15644,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1794}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18308","reported_seq":18467,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.769862+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:49.769862+0000","last_peered":"2024-09-16T04:57:49.769862+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:49.769862+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":1552,"num_object_clones":0,"num_object_copies":3104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14963,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1847}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18397","reported_seq":18623,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.736277+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:49.736277+0000","last_peered":"2024-09-16T04:57:49.736277+0000","last_clean":"2024-09-16T04:57:49.736277+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:49.736277+0000","last_undegraded":"2024-09-16T04:57:49.736277+0000","last_fullsized":"2024-09-16T04:57:49.736277+0000","mapping_epoch":34,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6631424,"num_objects":1619,"num_object_clones":0,"num_object_copies":3238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15467,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18483","reported_seq":18722,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.573052+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:49.573052+0000","last_peered":"2024-09-16T04:57:49.573052+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:49.573052+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6545408,"num_objects":1598,"num_object_clones":0,"num_object_copies":3196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15234,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1723}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18556","reported_seq":18696,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.794224+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:49.794224+0000","last_peered":"2024-09-16T04:57:49.794224+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:49.794224+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6807552,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15211,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1803}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18295","reported_seq":25151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.590040+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:49.590040+0000","last_peered":"2024-09-16T04:57:49.590040+0000","last_clean":"2024-09-16T04:57:49.590040+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:49.590040+0000","last_undegraded":"2024-09-16T04:57:49.590040+0000","last_fullsized":"2024-09-16T04:57:49.590040+0000","mapping_epoch":50,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:50.666501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6898898,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18267,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18369","reported_seq":25236,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.614666+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:49.614666+0000","last_peered":"2024-09-16T04:57:49.614666+0000","last_clean":"2024-09-16T04:57:49.614666+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:49.614666+0000","last_undegraded":"2024-09-16T04:57:49.614666+0000","last_fullsized":"2024-09-16T04:57:49.614666+0000","mapping_epoch":46,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6590464,"num_objects":1609,"num_object_clones":0,"num_object_copies":3218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15092,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18279","reported_seq":18387,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.599656+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:49.599656+0000","last_peered":"2024-09-16T04:57:49.599656+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:49.599656+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6541929,"num_objects":1598,"num_object_clones":0,"num_object_copies":3196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18278,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1809}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18828","reported_seq":18941,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.545938+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:49.545938+0000","last_peered":"2024-09-16T04:57:49.545938+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:49.545938+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6981435,"num_objects":1707,"num_object_clones":0,"num_object_copies":3414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18825,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18535","reported_seq":25339,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.374701+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:49.374701+0000","last_peered":"2024-09-16T04:57:49.374701+0000","last_clean":"2024-09-16T04:57:49.374701+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:49.374701+0000","last_undegraded":"2024-09-16T04:57:49.374701+0000","last_fullsized":"2024-09-16T04:57:49.374701+0000","mapping_epoch":48,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6811648,"num_objects":1663,"num_object_clones":0,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15289,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18745","reported_seq":18896,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.816844+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:49.816844+0000","last_peered":"2024-09-16T04:57:49.816844+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:49.816844+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6845033,"num_objects":1672,"num_object_clones":0,"num_object_copies":3344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15514,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":107061495,"num_objects":26145,"num_object_clones":0,"num_object_copies":52290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26145,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260485,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11185,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":381165568,"data_stored":381091323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161007,"ondisk_log_size":161007,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548856,"kb_used_data":147880,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93822984,"statfs":{"total":96636764160,"available":96074735616,"internally_reserved":0,"allocated":151429120,"data_stored":151135473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":410578601},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,1,1,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593827,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516376,"kb_used_data":79300,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93855464,"statfs":{"total":96636764160,"available":96107995136,"internally_reserved":0,"allocated":81203200,"data_stored":80853889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":447540910},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543404,"kb_used_data":146648,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93828436,"statfs":{"total":96636764160,"available":96080318464,"internally_reserved":0,"allocated":150167552,"data_stored":149877977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":149622784,"data_stored":149619305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80637952,"data_stored":80574737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":150904832,"data_stored":150897281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:51.997 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:52.701 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:52.701 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:52.756 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-09-16T04:57:52.504455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":107061514,"num_objects":26147,"num_object_clones":0,"num_object_copies":52294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26147,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260487,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11186,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161010,"ondisk_log_size":161010,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1608636,"kb_used_data":373828,"kb_used_omap":12,"kb_used_meta":1234739,"kb_avail":281506884,"statfs":{"total":289910292480,"available":288263049216,"internally_reserved":0,"allocated":382799872,"data_stored":381867339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1264372737},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,3,7,8,10,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19296256,"num_objects":-4711,"num_object_clones":0,"num_object_copies":-9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4711,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009957"},"pg_stats":[{"pgid":"2.f","version":"51'18140","reported_seq":20641,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.798458+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:49.798458+0000","last_peered":"2024-09-16T04:57:49.798458+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:49.798458+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6545408,"num_objects":1598,"num_object_clones":0,"num_object_copies":3196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1206,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-538}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18399","reported_seq":18526,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.621442+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:49.621442+0000","last_peered":"2024-09-16T04:57:49.621442+0000","last_clean":"2024-09-16T04:57:49.621442+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:49.621442+0000","last_undegraded":"2024-09-16T04:57:49.621442+0000","last_fullsized":"2024-09-16T04:57:49.621442+0000","mapping_epoch":35,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6533120,"num_objects":1595,"num_object_clones":0,"num_object_copies":3190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15368,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18279","reported_seq":18426,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.647081+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:49.647081+0000","last_peered":"2024-09-16T04:57:49.647081+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:49.647081+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6463512,"num_objects":1579,"num_object_clones":0,"num_object_copies":3158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15018,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1740}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18652","reported_seq":18733,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.614484+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:49.614484+0000","last_peered":"2024-09-16T04:57:49.614484+0000","last_clean":"2024-09-16T04:57:49.614484+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:49.614484+0000","last_undegraded":"2024-09-16T04:57:49.614484+0000","last_fullsized":"2024-09-16T04:57:49.614484+0000","mapping_epoch":35,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6930432,"num_objects":1692,"num_object_clones":0,"num_object_copies":3384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18652,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18491","reported_seq":18720,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.798920+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:49.798920+0000","last_peered":"2024-09-16T04:57:49.798920+0000","last_clean":"2024-09-16T04:57:49.798920+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:49.798920+0000","last_undegraded":"2024-09-16T04:57:49.798920+0000","last_fullsized":"2024-09-16T04:57:49.798920+0000","mapping_epoch":34,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6770688,"num_objects":1653,"num_object_clones":0,"num_object_copies":3306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15523,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18651","reported_seq":18794,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.793426+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:49.793426+0000","last_peered":"2024-09-16T04:57:49.793426+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:49.793426+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6807552,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15644,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1794}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18308","reported_seq":18467,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.769862+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:49.769862+0000","last_peered":"2024-09-16T04:57:49.769862+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:49.769862+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":1552,"num_object_clones":0,"num_object_copies":3104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14963,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1847}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18397","reported_seq":18623,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.736277+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:49.736277+0000","last_peered":"2024-09-16T04:57:49.736277+0000","last_clean":"2024-09-16T04:57:49.736277+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:49.736277+0000","last_undegraded":"2024-09-16T04:57:49.736277+0000","last_fullsized":"2024-09-16T04:57:49.736277+0000","mapping_epoch":34,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6631424,"num_objects":1619,"num_object_clones":0,"num_object_copies":3238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15467,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18483","reported_seq":18722,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.573052+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:49.573052+0000","last_peered":"2024-09-16T04:57:49.573052+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:49.573052+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6545408,"num_objects":1598,"num_object_clones":0,"num_object_copies":3196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15234,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1723}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18556","reported_seq":18696,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.794224+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:49.794224+0000","last_peered":"2024-09-16T04:57:49.794224+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:49.794224+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6807552,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15211,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1803}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18295","reported_seq":25151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.590040+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:49.590040+0000","last_peered":"2024-09-16T04:57:49.590040+0000","last_clean":"2024-09-16T04:57:49.590040+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:49.590040+0000","last_undegraded":"2024-09-16T04:57:49.590040+0000","last_fullsized":"2024-09-16T04:57:49.590040+0000","mapping_epoch":50,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:50.666501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6898898,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18267,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18369","reported_seq":25236,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.614666+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:49.614666+0000","last_peered":"2024-09-16T04:57:49.614666+0000","last_clean":"2024-09-16T04:57:49.614666+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:49.614666+0000","last_undegraded":"2024-09-16T04:57:49.614666+0000","last_fullsized":"2024-09-16T04:57:49.614666+0000","mapping_epoch":46,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6590464,"num_objects":1609,"num_object_clones":0,"num_object_copies":3218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15092,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18279","reported_seq":18387,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.599656+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:49.599656+0000","last_peered":"2024-09-16T04:57:49.599656+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:49.599656+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6541929,"num_objects":1598,"num_object_clones":0,"num_object_copies":3196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18278,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1809}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18828","reported_seq":18941,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.545938+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:49.545938+0000","last_peered":"2024-09-16T04:57:49.545938+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:49.545938+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6981435,"num_objects":1707,"num_object_clones":0,"num_object_copies":3414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18825,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18535","reported_seq":25339,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.374701+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:49.374701+0000","last_peered":"2024-09-16T04:57:49.374701+0000","last_clean":"2024-09-16T04:57:49.374701+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:49.374701+0000","last_undegraded":"2024-09-16T04:57:49.374701+0000","last_fullsized":"2024-09-16T04:57:49.374701+0000","mapping_epoch":48,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6811648,"num_objects":1663,"num_object_clones":0,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15289,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18745","reported_seq":18896,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.816844+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:49.816844+0000","last_peered":"2024-09-16T04:57:49.816844+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:49.816844+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6845033,"num_objects":1672,"num_object_clones":0,"num_object_copies":3344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15514,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":107061495,"num_objects":26145,"num_object_clones":0,"num_object_copies":52290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26145,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260485,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11185,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":381165568,"data_stored":381091323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161007,"ondisk_log_size":161007,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548856,"kb_used_data":147880,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93822984,"statfs":{"total":96636764160,"available":96074735616,"internally_reserved":0,"allocated":151429120,"data_stored":151135473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":410578601},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,1,1,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593827,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516376,"kb_used_data":79300,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93855464,"statfs":{"total":96636764160,"available":96107995136,"internally_reserved":0,"allocated":81203200,"data_stored":80853889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":447540910},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543404,"kb_used_data":146648,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93828436,"statfs":{"total":96636764160,"available":96080318464,"internally_reserved":0,"allocated":150167552,"data_stored":149877977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":149622784,"data_stored":149619305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80637952,"data_stored":80574737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":150904832,"data_stored":150897281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:52.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:57:52.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'18140", 'reported_seq': 20641, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:49.798458+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:57:49.798458+0000', 'last_peered': '2024-09-16T04:57:49.798458+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:57:49.798458+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'8100", 'ondisk_log_start': "47'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6545408, 'num_objects': 1598, 'num_object_clones': 0, 'num_object_copies': 3196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2136, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1598, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18140, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1206, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -538}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:52.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:57:52.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'18279", 'reported_seq': 18426, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:49.647081+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:57:49.647081+0000', 'last_peered': '2024-09-16T04:57:49.647081+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:57:49.647081+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'8200", 'ondisk_log_start': "47'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6463512, 'num_objects': 1579, 'num_object_clones': 0, 'num_object_copies': 3158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1579, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15018, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1740}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:52.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:57:52.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'18651", 'reported_seq': 18794, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:49.793426+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:57:49.793426+0000', 'last_peered': '2024-09-16T04:57:49.793426+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:57:49.793426+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'8600", 'ondisk_log_start': "47'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6807552, 'num_objects': 1662, 'num_object_clones': 0, 'num_object_copies': 3324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1662, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15644, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1794}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:52.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:57:52.758 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'18308", 'reported_seq': 18467, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:49.769862+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:57:49.769862+0000', 'last_peered': '2024-09-16T04:57:49.769862+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:57:49.769862+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'8300", 'ondisk_log_start': "47'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6356992, 'num_objects': 1552, 'num_object_clones': 0, 'num_object_copies': 3104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1552, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14963, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1847}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:52.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:57:52.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'18483", 'reported_seq': 18722, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:49.573052+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:57:49.573052+0000', 'last_peered': '2024-09-16T04:57:49.573052+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:57:49.573052+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'8400", 'ondisk_log_start': "47'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6545408, 'num_objects': 1598, 'num_object_clones': 0, 'num_object_copies': 3196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1598, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15234, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1723}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:52.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:57:52.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'18556", 'reported_seq': 18696, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:49.794224+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:57:49.794224+0000', 'last_peered': '2024-09-16T04:57:49.794224+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:57:49.794224+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'8500", 'ondisk_log_start': "47'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6807552, 'num_objects': 1662, 'num_object_clones': 0, 'num_object_copies': 3324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1662, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15211, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1803}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:52.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:57:52.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'18279", 'reported_seq': 18387, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:49.599656+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:57:49.599656+0000', 'last_peered': '2024-09-16T04:57:49.599656+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:57:49.599656+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'8200", 'ondisk_log_start': "47'8200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6541929, 'num_objects': 1598, 'num_object_clones': 0, 'num_object_copies': 3196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1598, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18278, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1809}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:52.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:57:52.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'18828", 'reported_seq': 18941, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:49.545938+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:57:49.545938+0000', 'last_peered': '2024-09-16T04:57:49.545938+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:57:49.545938+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'8800", 'ondisk_log_start': "47'8800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6981435, 'num_objects': 1707, 'num_object_clones': 0, 'num_object_copies': 3414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1707, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18825, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1807}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:52.760 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:57:52.760 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'18745", 'reported_seq': 18896, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:49.816844+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:57:49.816844+0000', 'last_peered': '2024-09-16T04:57:49.816844+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:57:49.816844+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'8700", 'ondisk_log_start': "47'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6845033, 'num_objects': 1672, 'num_object_clones': 0, 'num_object_copies': 3344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15514, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1852}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:52.760 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:57:53.794 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:53.794 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29557/52294 objects degraded (56.521%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":621,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":26147,"data_bytes":107061514,"bytes_used":1647243264,"bytes_avail":288263049216,"bytes_total":289910292480,"degraded_objects":29557,"degraded_total":52294,"degraded_ratio":0.56520824568784178,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":174,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":392},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:57:53.852 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:57:53.852 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:54.714 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:54.764 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-16T04:57:54.504883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":106443018,"num_objects":25996,"num_object_clones":0,"num_object_copies":51992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25996,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260638,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11186,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161061,"ondisk_log_size":161061,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1611260,"kb_used_data":366728,"kb_used_omap":12,"kb_used_meta":1244467,"kb_avail":281504260,"statfs":{"total":289910292480,"available":288260362240,"internally_reserved":0,"allocated":375529472,"data_stored":374596939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1274334209},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,4,4,9,9,11,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19914752,"num_objects":-4862,"num_object_clones":0,"num_object_copies":-9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4862,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009897"},"pg_stats":[{"pgid":"2.f","version":"51'18140","reported_seq":20641,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.798458+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:49.798458+0000","last_peered":"2024-09-16T04:57:49.798458+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:49.798458+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8100","ondisk_log_start":"47'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6545408,"num_objects":1598,"num_object_clones":0,"num_object_copies":3196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1206,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-538}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18399","reported_seq":18526,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.621442+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:49.621442+0000","last_peered":"2024-09-16T04:57:49.621442+0000","last_clean":"2024-09-16T04:57:49.621442+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:49.621442+0000","last_undegraded":"2024-09-16T04:57:49.621442+0000","last_fullsized":"2024-09-16T04:57:49.621442+0000","mapping_epoch":35,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6533120,"num_objects":1595,"num_object_clones":0,"num_object_copies":3190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15368,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18279","reported_seq":18426,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.647081+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:49.647081+0000","last_peered":"2024-09-16T04:57:49.647081+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:49.647081+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6463512,"num_objects":1579,"num_object_clones":0,"num_object_copies":3158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15018,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1740}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18652","reported_seq":18733,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.614484+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:49.614484+0000","last_peered":"2024-09-16T04:57:49.614484+0000","last_clean":"2024-09-16T04:57:49.614484+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:49.614484+0000","last_undegraded":"2024-09-16T04:57:49.614484+0000","last_fullsized":"2024-09-16T04:57:49.614484+0000","mapping_epoch":35,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6930432,"num_objects":1692,"num_object_clones":0,"num_object_copies":3384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18652,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18491","reported_seq":18720,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.798920+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:49.798920+0000","last_peered":"2024-09-16T04:57:49.798920+0000","last_clean":"2024-09-16T04:57:49.798920+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:49.798920+0000","last_undegraded":"2024-09-16T04:57:49.798920+0000","last_fullsized":"2024-09-16T04:57:49.798920+0000","mapping_epoch":34,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6770688,"num_objects":1653,"num_object_clones":0,"num_object_copies":3306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15523,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18651","reported_seq":18794,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.793426+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:49.793426+0000","last_peered":"2024-09-16T04:57:49.793426+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:49.793426+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6807552,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15644,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1794}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18308","reported_seq":18467,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.769862+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:49.769862+0000","last_peered":"2024-09-16T04:57:49.769862+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:49.769862+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6356992,"num_objects":1552,"num_object_clones":0,"num_object_copies":3104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14963,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1847}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18397","reported_seq":18623,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.736277+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:49.736277+0000","last_peered":"2024-09-16T04:57:49.736277+0000","last_clean":"2024-09-16T04:57:49.736277+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:49.736277+0000","last_undegraded":"2024-09-16T04:57:49.736277+0000","last_fullsized":"2024-09-16T04:57:49.736277+0000","mapping_epoch":34,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6631424,"num_objects":1619,"num_object_clones":0,"num_object_copies":3238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15467,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18483","reported_seq":18722,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.573052+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:49.573052+0000","last_peered":"2024-09-16T04:57:49.573052+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:49.573052+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6545408,"num_objects":1598,"num_object_clones":0,"num_object_copies":3196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15234,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1723}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18556","reported_seq":18696,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.794224+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:49.794224+0000","last_peered":"2024-09-16T04:57:49.794224+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:49.794224+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6807552,"num_objects":1662,"num_object_clones":0,"num_object_copies":3324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15211,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1803}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18295","reported_seq":25151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.590040+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:49.590040+0000","last_peered":"2024-09-16T04:57:49.590040+0000","last_clean":"2024-09-16T04:57:49.590040+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:49.590040+0000","last_undegraded":"2024-09-16T04:57:49.590040+0000","last_fullsized":"2024-09-16T04:57:49.590040+0000","mapping_epoch":50,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:50.666501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6898898,"num_objects":1686,"num_object_clones":0,"num_object_copies":3372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18267,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18369","reported_seq":25236,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:49.614666+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:49.614666+0000","last_peered":"2024-09-16T04:57:49.614666+0000","last_clean":"2024-09-16T04:57:49.614666+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:49.614666+0000","last_undegraded":"2024-09-16T04:57:49.614666+0000","last_fullsized":"2024-09-16T04:57:49.614666+0000","mapping_epoch":46,"log_start":"47'8300","ondisk_log_start":"47'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6590464,"num_objects":1609,"num_object_clones":0,"num_object_copies":3218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15092,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18279","reported_seq":18387,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.599656+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:49.599656+0000","last_peered":"2024-09-16T04:57:49.599656+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:49.599656+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6541929,"num_objects":1598,"num_object_clones":0,"num_object_copies":3196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18278,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1809}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18828","reported_seq":18941,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.545938+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:49.545938+0000","last_peered":"2024-09-16T04:57:49.545938+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:49.545938+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6981435,"num_objects":1707,"num_object_clones":0,"num_object_copies":3414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18825,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1807}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18686","reported_seq":25490,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.389625+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:54.389625+0000","last_peered":"2024-09-16T04:57:54.389625+0000","last_clean":"2024-09-16T04:57:54.389625+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:54.389625+0000","last_undegraded":"2024-09-16T04:57:54.389625+0000","last_fullsized":"2024-09-16T04:57:54.389625+0000","mapping_epoch":48,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6193152,"num_objects":1512,"num_object_clones":0,"num_object_copies":3024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15440,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18745","reported_seq":18896,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:49.816844+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:49.816844+0000","last_peered":"2024-09-16T04:57:49.816844+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:49.816844+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6845033,"num_objects":1672,"num_object_clones":0,"num_object_copies":3344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15514,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":106442999,"num_objects":25994,"num_object_clones":0,"num_object_copies":51988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25994,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260636,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11185,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":373878784,"data_stored":373804539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161058,"ondisk_log_size":161058,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548856,"kb_used_data":147880,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93822984,"statfs":{"total":96636764160,"available":96074735616,"internally_reserved":0,"allocated":151429120,"data_stored":151135473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":410578601},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,1,1,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593828,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519000,"kb_used_data":72200,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93852840,"statfs":{"total":96636764160,"available":96105308160,"internally_reserved":0,"allocated":73932800,"data_stored":73583489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,3,2,7,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543404,"kb_used_data":146648,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93828436,"statfs":{"total":96636764160,"available":96080318464,"internally_reserved":0,"allocated":150167552,"data_stored":149877977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":149622784,"data_stored":149619305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73351168,"data_stored":73287953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":150904832,"data_stored":150897281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:57.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:58.608 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:58.678 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-16T04:57:58.511002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":97595658,"num_objects":23836,"num_object_clones":0,"num_object_copies":47672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29476,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23836,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":262798,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11267,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1591780,"kb_used_data":355572,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281523740,"statfs":{"total":289910292480,"available":288280309760,"internally_reserved":0,"allocated":364105728,"data_stored":363173195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1265814529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,5,2,9,8,9,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18997248,"num_objects":-4638,"num_object_clones":0,"num_object_copies":-9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4638,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.014182"},"pg_stats":[{"pgid":"2.f","version":"51'18290","reported_seq":20953,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.849963+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:54.849963+0000","last_peered":"2024-09-16T04:57:54.849963+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:54.849963+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5931008,"num_objects":1448,"num_object_clones":0,"num_object_copies":2896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18290,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1287,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18536","reported_seq":18663,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.646302+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:54.646302+0000","last_peered":"2024-09-16T04:57:54.646302+0000","last_clean":"2024-09-16T04:57:54.646302+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:54.646302+0000","last_undegraded":"2024-09-16T04:57:54.646302+0000","last_fullsized":"2024-09-16T04:57:54.646302+0000","mapping_epoch":35,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5971968,"num_objects":1458,"num_object_clones":0,"num_object_copies":2916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15505,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18413","reported_seq":18563,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.632056+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:54.632056+0000","last_peered":"2024-09-16T04:57:54.632056+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:54.632056+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5914648,"num_objects":1445,"num_object_clones":0,"num_object_copies":2890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1445,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15152,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1874}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18802","reported_seq":18883,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.561346+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:54.561346+0000","last_peered":"2024-09-16T04:57:54.561346+0000","last_clean":"2024-09-16T04:57:54.561346+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:54.561346+0000","last_undegraded":"2024-09-16T04:57:54.561346+0000","last_fullsized":"2024-09-16T04:57:54.561346+0000","mapping_epoch":35,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6316032,"num_objects":1542,"num_object_clones":0,"num_object_copies":3084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18802,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18602","reported_seq":18831,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.777329+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:54.777329+0000","last_peered":"2024-09-16T04:57:54.777329+0000","last_clean":"2024-09-16T04:57:54.777329+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:54.777329+0000","last_undegraded":"2024-09-16T04:57:54.777329+0000","last_fullsized":"2024-09-16T04:57:54.777329+0000","mapping_epoch":34,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6316032,"num_objects":1542,"num_object_clones":0,"num_object_copies":3084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15634,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18800","reported_seq":18942,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.795968+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:54.795968+0000","last_peered":"2024-09-16T04:57:54.795968+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:54.795968+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6197248,"num_objects":1513,"num_object_clones":0,"num_object_copies":3026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15793,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1943}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18429","reported_seq":18588,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.829467+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:54.829467+0000","last_peered":"2024-09-16T04:57:54.829467+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:54.829467+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5861376,"num_objects":1431,"num_object_clones":0,"num_object_copies":2862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15084,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18531","reported_seq":18757,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.766863+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:54.766863+0000","last_peered":"2024-09-16T04:57:54.766863+0000","last_clean":"2024-09-16T04:57:54.766863+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:54.766863+0000","last_undegraded":"2024-09-16T04:57:54.766863+0000","last_fullsized":"2024-09-16T04:57:54.766863+0000","mapping_epoch":34,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6082560,"num_objects":1485,"num_object_clones":0,"num_object_copies":2970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15601,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18639","reported_seq":18878,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.620897+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:54.620897+0000","last_peered":"2024-09-16T04:57:54.620897+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:54.620897+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5906432,"num_objects":1442,"num_object_clones":0,"num_object_copies":2884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15390,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1879}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18691","reported_seq":18829,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.795715+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:54.795715+0000","last_peered":"2024-09-16T04:57:54.795715+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:54.795715+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6254592,"num_objects":1527,"num_object_clones":0,"num_object_copies":3054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15346,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1938}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18445","reported_seq":25301,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.584903+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:54.584903+0000","last_peered":"2024-09-16T04:57:54.584903+0000","last_clean":"2024-09-16T04:57:54.584903+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:54.584903+0000","last_undegraded":"2024-09-16T04:57:54.584903+0000","last_fullsized":"2024-09-16T04:57:54.584903+0000","mapping_epoch":50,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:56.632563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6284498,"num_objects":1536,"num_object_clones":0,"num_object_copies":3072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18417,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18527","reported_seq":25394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.580971+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:54.580971+0000","last_peered":"2024-09-16T04:57:54.580971+0000","last_clean":"2024-09-16T04:57:54.580971+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:54.580971+0000","last_undegraded":"2024-09-16T04:57:54.580971+0000","last_fullsized":"2024-09-16T04:57:54.580971+0000","mapping_epoch":46,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5943296,"num_objects":1451,"num_object_clones":0,"num_object_copies":2902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15250,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18425","reported_seq":18528,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.543477+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:54.543477+0000","last_peered":"2024-09-16T04:57:54.543477+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:54.543477+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5943913,"num_objects":1452,"num_object_clones":0,"num_object_copies":2904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18424,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1955}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18990","reported_seq":19103,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.646904+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:54.646904+0000","last_peered":"2024-09-16T04:57:54.646904+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:54.646904+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6317883,"num_objects":1545,"num_object_clones":0,"num_object_copies":3090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18987,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1969}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18686","reported_seq":25490,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.389625+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:54.389625+0000","last_peered":"2024-09-16T04:57:54.389625+0000","last_clean":"2024-09-16T04:57:54.389625+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:54.389625+0000","last_undegraded":"2024-09-16T04:57:54.389625+0000","last_fullsized":"2024-09-16T04:57:54.389625+0000","mapping_epoch":48,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6193152,"num_objects":1512,"num_object_clones":0,"num_object_copies":3024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15440,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18912","reported_seq":19063,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.769044+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:54.769044+0000","last_peered":"2024-09-16T04:57:54.769044+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:54.769044+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6161001,"num_objects":1505,"num_object_clones":0,"num_object_copies":3010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15681,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2019}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":97595639,"num_objects":23834,"num_object_clones":0,"num_object_copies":47668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29476,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23834,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":262796,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11266,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":362311680,"data_stored":362237435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530688,"kb_used_data":142192,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93841152,"statfs":{"total":96636764160,"available":96093339648,"internally_reserved":0,"allocated":145604608,"data_stored":145310961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":397799081},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1,0,1,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593828,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519000,"kb_used_data":72200,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93852840,"statfs":{"total":96636764160,"available":96105308160,"internally_reserved":0,"allocated":73932800,"data_stored":73583489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,3,2,7,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542092,"kb_used_data":141180,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93829748,"statfs":{"total":96636764160,"available":96081661952,"internally_reserved":0,"allocated":144568320,"data_stored":144278745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143933440,"data_stored":143929961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73351168,"data_stored":73287953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":145027072,"data_stored":145019521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:58.680 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:57:59.449 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:57:59.451 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:57:59.516 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-16T04:57:58.511002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":97595658,"num_objects":23836,"num_object_clones":0,"num_object_copies":47672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29476,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23836,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":262798,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11267,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1591780,"kb_used_data":355572,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281523740,"statfs":{"total":289910292480,"available":288280309760,"internally_reserved":0,"allocated":364105728,"data_stored":363173195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1265814529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,5,2,9,8,9,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18997248,"num_objects":-4638,"num_object_clones":0,"num_object_copies":-9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4638,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.014182"},"pg_stats":[{"pgid":"2.f","version":"51'18290","reported_seq":20953,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.849963+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:54.849963+0000","last_peered":"2024-09-16T04:57:54.849963+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:54.849963+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8200","ondisk_log_start":"47'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5931008,"num_objects":1448,"num_object_clones":0,"num_object_copies":2896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18290,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1287,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-607}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18536","reported_seq":18663,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.646302+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:54.646302+0000","last_peered":"2024-09-16T04:57:54.646302+0000","last_clean":"2024-09-16T04:57:54.646302+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:54.646302+0000","last_undegraded":"2024-09-16T04:57:54.646302+0000","last_fullsized":"2024-09-16T04:57:54.646302+0000","mapping_epoch":35,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5971968,"num_objects":1458,"num_object_clones":0,"num_object_copies":2916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15505,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18413","reported_seq":18563,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.632056+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:54.632056+0000","last_peered":"2024-09-16T04:57:54.632056+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:54.632056+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5914648,"num_objects":1445,"num_object_clones":0,"num_object_copies":2890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1445,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15152,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1874}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18802","reported_seq":18883,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.561346+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:54.561346+0000","last_peered":"2024-09-16T04:57:54.561346+0000","last_clean":"2024-09-16T04:57:54.561346+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:54.561346+0000","last_undegraded":"2024-09-16T04:57:54.561346+0000","last_fullsized":"2024-09-16T04:57:54.561346+0000","mapping_epoch":35,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6316032,"num_objects":1542,"num_object_clones":0,"num_object_copies":3084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18802,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18602","reported_seq":18831,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.777329+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:54.777329+0000","last_peered":"2024-09-16T04:57:54.777329+0000","last_clean":"2024-09-16T04:57:54.777329+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:54.777329+0000","last_undegraded":"2024-09-16T04:57:54.777329+0000","last_fullsized":"2024-09-16T04:57:54.777329+0000","mapping_epoch":34,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6316032,"num_objects":1542,"num_object_clones":0,"num_object_copies":3084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15634,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18800","reported_seq":18942,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.795968+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:54.795968+0000","last_peered":"2024-09-16T04:57:54.795968+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:54.795968+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6197248,"num_objects":1513,"num_object_clones":0,"num_object_copies":3026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15793,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1943}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18429","reported_seq":18588,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.829467+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:54.829467+0000","last_peered":"2024-09-16T04:57:54.829467+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:54.829467+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5861376,"num_objects":1431,"num_object_clones":0,"num_object_copies":2862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15084,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1968}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18531","reported_seq":18757,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.766863+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:54.766863+0000","last_peered":"2024-09-16T04:57:54.766863+0000","last_clean":"2024-09-16T04:57:54.766863+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:54.766863+0000","last_undegraded":"2024-09-16T04:57:54.766863+0000","last_fullsized":"2024-09-16T04:57:54.766863+0000","mapping_epoch":34,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6082560,"num_objects":1485,"num_object_clones":0,"num_object_copies":2970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15601,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18639","reported_seq":18878,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.620897+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:54.620897+0000","last_peered":"2024-09-16T04:57:54.620897+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:54.620897+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5906432,"num_objects":1442,"num_object_clones":0,"num_object_copies":2884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15390,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1879}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18691","reported_seq":18829,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.795715+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:54.795715+0000","last_peered":"2024-09-16T04:57:54.795715+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:54.795715+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6254592,"num_objects":1527,"num_object_clones":0,"num_object_copies":3054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15346,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1938}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18445","reported_seq":25301,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.584903+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:54.584903+0000","last_peered":"2024-09-16T04:57:54.584903+0000","last_clean":"2024-09-16T04:57:54.584903+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:54.584903+0000","last_undegraded":"2024-09-16T04:57:54.584903+0000","last_fullsized":"2024-09-16T04:57:54.584903+0000","mapping_epoch":50,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:57:56.632563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6284498,"num_objects":1536,"num_object_clones":0,"num_object_copies":3072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18417,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710336+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710336+0000","last_peered":"2024-09-16T04:56:42.710336+0000","last_clean":"2024-09-16T04:56:42.710336+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710336+0000","last_undegraded":"2024-09-16T04:56:42.710336+0000","last_fullsized":"2024-09-16T04:56:42.710336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18527","reported_seq":25394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.580971+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:54.580971+0000","last_peered":"2024-09-16T04:57:54.580971+0000","last_clean":"2024-09-16T04:57:54.580971+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:54.580971+0000","last_undegraded":"2024-09-16T04:57:54.580971+0000","last_fullsized":"2024-09-16T04:57:54.580971+0000","mapping_epoch":46,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5943296,"num_objects":1451,"num_object_clones":0,"num_object_copies":2902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15250,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18425","reported_seq":18528,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.543477+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:54.543477+0000","last_peered":"2024-09-16T04:57:54.543477+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:54.543477+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5943913,"num_objects":1452,"num_object_clones":0,"num_object_copies":2904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18424,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1955}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711070+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711070+0000","last_peered":"2024-09-16T04:56:42.711070+0000","last_clean":"2024-09-16T04:56:42.711070+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711070+0000","last_undegraded":"2024-09-16T04:56:42.711070+0000","last_fullsized":"2024-09-16T04:56:42.711070+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'18990","reported_seq":19103,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.646904+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:54.646904+0000","last_peered":"2024-09-16T04:57:54.646904+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:54.646904+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6317883,"num_objects":1545,"num_object_clones":0,"num_object_copies":3090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18987,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1969}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18686","reported_seq":25490,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:54.389625+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:54.389625+0000","last_peered":"2024-09-16T04:57:54.389625+0000","last_clean":"2024-09-16T04:57:54.389625+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:54.389625+0000","last_undegraded":"2024-09-16T04:57:54.389625+0000","last_fullsized":"2024-09-16T04:57:54.389625+0000","mapping_epoch":48,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6193152,"num_objects":1512,"num_object_clones":0,"num_object_copies":3024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15440,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'18912","reported_seq":19063,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:54.769044+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:54.769044+0000","last_peered":"2024-09-16T04:57:54.769044+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:54.769044+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6161001,"num_objects":1505,"num_object_clones":0,"num_object_copies":3010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15681,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2019}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":97595639,"num_objects":23834,"num_object_clones":0,"num_object_copies":47668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29476,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23834,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":262796,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11266,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":362311680,"data_stored":362237435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530688,"kb_used_data":142192,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93841152,"statfs":{"total":96636764160,"available":96093339648,"internally_reserved":0,"allocated":145604608,"data_stored":145310961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":397799081},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1,0,1,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593828,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519000,"kb_used_data":72200,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93852840,"statfs":{"total":96636764160,"available":96105308160,"internally_reserved":0,"allocated":73932800,"data_stored":73583489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,3,2,7,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":542092,"kb_used_data":141180,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93829748,"statfs":{"total":96636764160,"available":96081661952,"internally_reserved":0,"allocated":144568320,"data_stored":144278745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143933440,"data_stored":143929961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73351168,"data_stored":73287953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":145027072,"data_stored":145019521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:57:59.517 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:57:59.517 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'18290", 'reported_seq': 20953, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:54.849963+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:57:54.849963+0000', 'last_peered': '2024-09-16T04:57:54.849963+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:57:54.849963+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'8200", 'ondisk_log_start': "47'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5931008, 'num_objects': 1448, 'num_object_clones': 0, 'num_object_copies': 2896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2055, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18290, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1287, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -607}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:59.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:57:59.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'18413", 'reported_seq': 18563, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:54.632056+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:57:54.632056+0000', 'last_peered': '2024-09-16T04:57:54.632056+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:57:54.632056+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'8400", 'ondisk_log_start': "47'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5914648, 'num_objects': 1445, 'num_object_clones': 0, 'num_object_copies': 2890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1445, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15152, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1874}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:59.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:57:59.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'18800", 'reported_seq': 18942, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:54.795968+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:57:54.795968+0000', 'last_peered': '2024-09-16T04:57:54.795968+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:57:54.795968+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'8700", 'ondisk_log_start': "47'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6197248, 'num_objects': 1513, 'num_object_clones': 0, 'num_object_copies': 3026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1513, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15793, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1943}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:59.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:57:59.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'18429", 'reported_seq': 18588, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:54.829467+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:57:54.829467+0000', 'last_peered': '2024-09-16T04:57:54.829467+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:57:54.829467+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'8400", 'ondisk_log_start': "47'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5861376, 'num_objects': 1431, 'num_object_clones': 0, 'num_object_copies': 2862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1431, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15084, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1968}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:59.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:57:59.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'18639", 'reported_seq': 18878, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:54.620897+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:57:54.620897+0000', 'last_peered': '2024-09-16T04:57:54.620897+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:57:54.620897+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'8600", 'ondisk_log_start': "47'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5906432, 'num_objects': 1442, 'num_object_clones': 0, 'num_object_copies': 2884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15390, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1879}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:59.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:57:59.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'18691", 'reported_seq': 18829, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:54.795715+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:57:54.795715+0000', 'last_peered': '2024-09-16T04:57:54.795715+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:57:54.795715+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'8600", 'ondisk_log_start': "47'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6254592, 'num_objects': 1527, 'num_object_clones': 0, 'num_object_copies': 3054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1527, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15346, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1938}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:59.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:57:59.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'18425", 'reported_seq': 18528, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:54.543477+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:57:54.543477+0000', 'last_peered': '2024-09-16T04:57:54.543477+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:57:54.543477+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'8400", 'ondisk_log_start': "47'8400", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5943913, 'num_objects': 1452, 'num_object_clones': 0, 'num_object_copies': 2904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1452, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18424, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1955}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:59.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:57:59.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'18990", 'reported_seq': 19103, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:54.646904+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:57:54.646904+0000', 'last_peered': '2024-09-16T04:57:54.646904+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:57:54.646904+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'8900", 'ondisk_log_start': "47'8900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6317883, 'num_objects': 1545, 'num_object_clones': 0, 'num_object_copies': 3090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1545, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18987, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1969}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:57:59.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:57:59.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'18912", 'reported_seq': 19063, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:54.769044+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:57:54.769044+0000', 'last_peered': '2024-09-16T04:57:54.769044+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:57:54.769044+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'8900", 'ondisk_log_start': "47'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6161001, 'num_objects': 1505, 'num_object_clones': 0, 'num_object_copies': 3010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1505, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15681, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2019}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:57:59.519 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:58:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:00.452 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29476/47672 objects degraded (61.831%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":628,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":23836,"data_bytes":97595658,"bytes_used":1629982720,"bytes_avail":288280309760,"bytes_total":289910292480,"degraded_objects":29476,"degraded_total":47672,"degraded_ratio":0.61830844101359284,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":386},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:58:00.510 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:58:00.510 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:01.331 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:01.332 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:01.375 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-16T04:58:00.513066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":87830794,"num_objects":21452,"num_object_clones":0,"num_object_copies":42904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265182,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11371,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161005,"ondisk_log_size":161005,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1572624,"kb_used_data":336412,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281542896,"statfs":{"total":289910292480,"available":288299925504,"internally_reserved":0,"allocated":344485888,"data_stored":343553355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1265814529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,11,9,12,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28762112,"num_objects":-7022,"num_object_clones":0,"num_object_copies":-14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7022,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011610"},"pg_stats":[{"pgid":"2.f","version":"51'18432","reported_seq":21303,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.819604+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:59.819604+0000","last_peered":"2024-09-16T04:57:59.819604+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:59.819604+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5349376,"num_objects":1306,"num_object_clones":0,"num_object_copies":2612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1391,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-645}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18686","reported_seq":18813,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.654541+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:59.654541+0000","last_peered":"2024-09-16T04:57:59.654541+0000","last_clean":"2024-09-16T04:57:59.654541+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:59.654541+0000","last_undegraded":"2024-09-16T04:57:59.654541+0000","last_fullsized":"2024-09-16T04:57:59.654541+0000","mapping_epoch":35,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5357568,"num_objects":1308,"num_object_clones":0,"num_object_copies":2616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15655,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18549","reported_seq":18699,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.603964+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:59.603964+0000","last_peered":"2024-09-16T04:57:59.603964+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:59.603964+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5357592,"num_objects":1309,"num_object_clones":0,"num_object_copies":2618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15288,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2010}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18939","reported_seq":19020,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.503701+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:59.503701+0000","last_peered":"2024-09-16T04:57:59.503701+0000","last_clean":"2024-09-16T04:57:59.503701+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:59.503701+0000","last_undegraded":"2024-09-16T04:57:59.503701+0000","last_fullsized":"2024-09-16T04:57:59.503701+0000","mapping_epoch":35,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5754880,"num_objects":1405,"num_object_clones":0,"num_object_copies":2810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18939,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18784","reported_seq":19013,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.800079+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:59.800079+0000","last_peered":"2024-09-16T04:57:59.800079+0000","last_clean":"2024-09-16T04:57:59.800079+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:59.800079+0000","last_undegraded":"2024-09-16T04:57:59.800079+0000","last_fullsized":"2024-09-16T04:57:59.800079+0000","mapping_epoch":34,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":1360,"num_object_clones":0,"num_object_copies":2720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15816,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18945","reported_seq":19087,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.855861+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:59.855861+0000","last_peered":"2024-09-16T04:57:59.855861+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:59.855861+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5603328,"num_objects":1368,"num_object_clones":0,"num_object_copies":2736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15938,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2088}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18574","reported_seq":18733,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.771595+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:59.771595+0000","last_peered":"2024-09-16T04:57:59.771595+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:59.771595+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267456,"num_objects":1286,"num_object_clones":0,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15229,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2113}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18679","reported_seq":18905,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.848328+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:59.848328+0000","last_peered":"2024-09-16T04:57:59.848328+0000","last_clean":"2024-09-16T04:57:59.848328+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:59.848328+0000","last_undegraded":"2024-09-16T04:57:59.848328+0000","last_fullsized":"2024-09-16T04:57:59.848328+0000","mapping_epoch":34,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5476352,"num_objects":1337,"num_object_clones":0,"num_object_copies":2674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15749,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18799","reported_seq":19038,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.636391+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:59.636391+0000","last_peered":"2024-09-16T04:57:59.636391+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:59.636391+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5251072,"num_objects":1282,"num_object_clones":0,"num_object_copies":2564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15550,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2039}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18845","reported_seq":18983,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.797877+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:59.797877+0000","last_peered":"2024-09-16T04:57:59.797877+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:59.797877+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5623808,"num_objects":1373,"num_object_clones":0,"num_object_copies":2746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15500,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2092}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18576","reported_seq":25432,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.592586+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:59.592586+0000","last_peered":"2024-09-16T04:57:59.592586+0000","last_clean":"2024-09-16T04:57:59.592586+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:59.592586+0000","last_undegraded":"2024-09-16T04:57:59.592586+0000","last_fullsized":"2024-09-16T04:57:59.592586+0000","mapping_epoch":50,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:02.539106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5747922,"num_objects":1405,"num_object_clones":0,"num_object_copies":2810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18548,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18682","reported_seq":25549,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.617507+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:59.617507+0000","last_peered":"2024-09-16T04:57:59.617507+0000","last_clean":"2024-09-16T04:57:59.617507+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:59.617507+0000","last_undegraded":"2024-09-16T04:57:59.617507+0000","last_fullsized":"2024-09-16T04:57:59.617507+0000","mapping_epoch":46,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":1296,"num_object_clones":0,"num_object_copies":2592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15405,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18559","reported_seq":18665,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.665599+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:59.665599+0000","last_peered":"2024-09-16T04:57:59.665599+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:59.665599+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5395049,"num_objects":1318,"num_object_clones":0,"num_object_copies":2636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18558,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2089}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19159","reported_seq":19272,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.619781+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:59.619781+0000","last_peered":"2024-09-16T04:57:59.619781+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:59.619781+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5625659,"num_objects":1376,"num_object_clones":0,"num_object_copies":2752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19156,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2138}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18816","reported_seq":25620,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.348613+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:59.348613+0000","last_peered":"2024-09-16T04:57:59.348613+0000","last_clean":"2024-09-16T04:57:59.348613+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:59.348613+0000","last_undegraded":"2024-09-16T04:57:59.348613+0000","last_fullsized":"2024-09-16T04:57:59.348613+0000","mapping_epoch":48,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5660672,"num_objects":1382,"num_object_clones":0,"num_object_copies":2764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15570,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19078","reported_seq":19229,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.761140+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:59.761140+0000","last_peered":"2024-09-16T04:57:59.761140+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:59.761140+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5481065,"num_objects":1339,"num_object_clones":0,"num_object_copies":2678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15847,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2185}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":87830775,"num_objects":21450,"num_object_clones":0,"num_object_copies":42900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21450,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265180,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11370,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":342728704,"data_stored":342654459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161002,"ondisk_log_size":161002,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528788,"kb_used_data":136128,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93843052,"statfs":{"total":96636764160,"available":96095285248,"internally_reserved":0,"allocated":139395072,"data_stored":139101425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":402058921},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,7,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593829,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516044,"kb_used_data":65084,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93855796,"statfs":{"total":96636764160,"available":96108335104,"internally_reserved":0,"allocated":66646016,"data_stored":66296705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527792,"kb_used_data":135200,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93844048,"statfs":{"total":96636764160,"available":96096305152,"internally_reserved":0,"allocated":138444800,"data_stored":138155225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,0,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":137809920,"data_stored":137806441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66068480,"data_stored":66005265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":138850304,"data_stored":138842753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:04.378 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:05.215 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:05.216 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:05.280 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-16T04:58:04.519883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":87830794,"num_objects":21452,"num_object_clones":0,"num_object_copies":42904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265182,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11371,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161005,"ondisk_log_size":161005,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1572624,"kb_used_data":336412,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281542896,"statfs":{"total":289910292480,"available":288299925504,"internally_reserved":0,"allocated":344485888,"data_stored":343553355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1265814529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,11,9,12,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19230720,"num_objects":-4695,"num_object_clones":0,"num_object_copies":-9390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4695,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.015428"},"pg_stats":[{"pgid":"2.f","version":"51'18432","reported_seq":21303,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.819604+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:59.819604+0000","last_peered":"2024-09-16T04:57:59.819604+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:59.819604+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5349376,"num_objects":1306,"num_object_clones":0,"num_object_copies":2612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1391,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-645}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18686","reported_seq":18813,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.654541+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:59.654541+0000","last_peered":"2024-09-16T04:57:59.654541+0000","last_clean":"2024-09-16T04:57:59.654541+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:59.654541+0000","last_undegraded":"2024-09-16T04:57:59.654541+0000","last_fullsized":"2024-09-16T04:57:59.654541+0000","mapping_epoch":35,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5357568,"num_objects":1308,"num_object_clones":0,"num_object_copies":2616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15655,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18549","reported_seq":18699,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.603964+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:59.603964+0000","last_peered":"2024-09-16T04:57:59.603964+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:59.603964+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5357592,"num_objects":1309,"num_object_clones":0,"num_object_copies":2618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15288,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2010}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18939","reported_seq":19020,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.503701+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:59.503701+0000","last_peered":"2024-09-16T04:57:59.503701+0000","last_clean":"2024-09-16T04:57:59.503701+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:59.503701+0000","last_undegraded":"2024-09-16T04:57:59.503701+0000","last_fullsized":"2024-09-16T04:57:59.503701+0000","mapping_epoch":35,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5754880,"num_objects":1405,"num_object_clones":0,"num_object_copies":2810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18939,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18784","reported_seq":19013,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.800079+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:59.800079+0000","last_peered":"2024-09-16T04:57:59.800079+0000","last_clean":"2024-09-16T04:57:59.800079+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:59.800079+0000","last_undegraded":"2024-09-16T04:57:59.800079+0000","last_fullsized":"2024-09-16T04:57:59.800079+0000","mapping_epoch":34,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":1360,"num_object_clones":0,"num_object_copies":2720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15816,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18945","reported_seq":19087,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.855861+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:59.855861+0000","last_peered":"2024-09-16T04:57:59.855861+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:59.855861+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5603328,"num_objects":1368,"num_object_clones":0,"num_object_copies":2736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15938,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2088}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18574","reported_seq":18733,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.771595+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:59.771595+0000","last_peered":"2024-09-16T04:57:59.771595+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:59.771595+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267456,"num_objects":1286,"num_object_clones":0,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15229,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2113}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18679","reported_seq":18905,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.848328+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:59.848328+0000","last_peered":"2024-09-16T04:57:59.848328+0000","last_clean":"2024-09-16T04:57:59.848328+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:59.848328+0000","last_undegraded":"2024-09-16T04:57:59.848328+0000","last_fullsized":"2024-09-16T04:57:59.848328+0000","mapping_epoch":34,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5476352,"num_objects":1337,"num_object_clones":0,"num_object_copies":2674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15749,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18799","reported_seq":19038,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.636391+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:59.636391+0000","last_peered":"2024-09-16T04:57:59.636391+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:59.636391+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5251072,"num_objects":1282,"num_object_clones":0,"num_object_copies":2564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15550,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2039}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18845","reported_seq":18983,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.797877+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:59.797877+0000","last_peered":"2024-09-16T04:57:59.797877+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:59.797877+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5623808,"num_objects":1373,"num_object_clones":0,"num_object_copies":2746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15500,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2092}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18576","reported_seq":25432,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.592586+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:59.592586+0000","last_peered":"2024-09-16T04:57:59.592586+0000","last_clean":"2024-09-16T04:57:59.592586+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:59.592586+0000","last_undegraded":"2024-09-16T04:57:59.592586+0000","last_fullsized":"2024-09-16T04:57:59.592586+0000","mapping_epoch":50,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:02.539106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5747922,"num_objects":1405,"num_object_clones":0,"num_object_copies":2810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18548,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18682","reported_seq":25549,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.617507+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:59.617507+0000","last_peered":"2024-09-16T04:57:59.617507+0000","last_clean":"2024-09-16T04:57:59.617507+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:59.617507+0000","last_undegraded":"2024-09-16T04:57:59.617507+0000","last_fullsized":"2024-09-16T04:57:59.617507+0000","mapping_epoch":46,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":1296,"num_object_clones":0,"num_object_copies":2592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15405,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18559","reported_seq":18665,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.665599+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:59.665599+0000","last_peered":"2024-09-16T04:57:59.665599+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:59.665599+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5395049,"num_objects":1318,"num_object_clones":0,"num_object_copies":2636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18558,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2089}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19159","reported_seq":19272,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.619781+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:59.619781+0000","last_peered":"2024-09-16T04:57:59.619781+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:59.619781+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5625659,"num_objects":1376,"num_object_clones":0,"num_object_copies":2752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19156,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2138}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18816","reported_seq":25620,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.348613+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:59.348613+0000","last_peered":"2024-09-16T04:57:59.348613+0000","last_clean":"2024-09-16T04:57:59.348613+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:59.348613+0000","last_undegraded":"2024-09-16T04:57:59.348613+0000","last_fullsized":"2024-09-16T04:57:59.348613+0000","mapping_epoch":48,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5660672,"num_objects":1382,"num_object_clones":0,"num_object_copies":2764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15570,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19078","reported_seq":19229,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.761140+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:59.761140+0000","last_peered":"2024-09-16T04:57:59.761140+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:59.761140+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5481065,"num_objects":1339,"num_object_clones":0,"num_object_copies":2678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15847,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2185}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":87830775,"num_objects":21450,"num_object_clones":0,"num_object_copies":42900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21450,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265180,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11370,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":342728704,"data_stored":342654459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161002,"ondisk_log_size":161002,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528788,"kb_used_data":136128,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93843052,"statfs":{"total":96636764160,"available":96095285248,"internally_reserved":0,"allocated":139395072,"data_stored":139101425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":402058921},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,7,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593829,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516044,"kb_used_data":65084,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93855796,"statfs":{"total":96636764160,"available":96108335104,"internally_reserved":0,"allocated":66646016,"data_stored":66296705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527792,"kb_used_data":135200,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93844048,"statfs":{"total":96636764160,"available":96096305152,"internally_reserved":0,"allocated":138444800,"data_stored":138155225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,0,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":137809920,"data_stored":137806441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66068480,"data_stored":66005265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":138850304,"data_stored":138842753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:05.282 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:06.032 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:06.033 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:06.083 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-16T04:58:04.519883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":87830794,"num_objects":21452,"num_object_clones":0,"num_object_copies":42904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265182,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11371,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161005,"ondisk_log_size":161005,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1572624,"kb_used_data":336412,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281542896,"statfs":{"total":289910292480,"available":288299925504,"internally_reserved":0,"allocated":344485888,"data_stored":343553355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1265814529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,11,9,12,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19230720,"num_objects":-4695,"num_object_clones":0,"num_object_copies":-9390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4695,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.015428"},"pg_stats":[{"pgid":"2.f","version":"51'18432","reported_seq":21303,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.819604+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:57:59.819604+0000","last_peered":"2024-09-16T04:57:59.819604+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:57:59.819604+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8400","ondisk_log_start":"47'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5349376,"num_objects":1306,"num_object_clones":0,"num_object_copies":2612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18432,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1391,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-645}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18686","reported_seq":18813,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.654541+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:57:59.654541+0000","last_peered":"2024-09-16T04:57:59.654541+0000","last_clean":"2024-09-16T04:57:59.654541+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:57:59.654541+0000","last_undegraded":"2024-09-16T04:57:59.654541+0000","last_fullsized":"2024-09-16T04:57:59.654541+0000","mapping_epoch":35,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5357568,"num_objects":1308,"num_object_clones":0,"num_object_copies":2616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15655,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18549","reported_seq":18699,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.603964+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:57:59.603964+0000","last_peered":"2024-09-16T04:57:59.603964+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:57:59.603964+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5357592,"num_objects":1309,"num_object_clones":0,"num_object_copies":2618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15288,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2010}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'18939","reported_seq":19020,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.503701+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:57:59.503701+0000","last_peered":"2024-09-16T04:57:59.503701+0000","last_clean":"2024-09-16T04:57:59.503701+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:57:59.503701+0000","last_undegraded":"2024-09-16T04:57:59.503701+0000","last_fullsized":"2024-09-16T04:57:59.503701+0000","mapping_epoch":35,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5754880,"num_objects":1405,"num_object_clones":0,"num_object_copies":2810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18939,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18784","reported_seq":19013,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.800079+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:57:59.800079+0000","last_peered":"2024-09-16T04:57:59.800079+0000","last_clean":"2024-09-16T04:57:59.800079+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:57:59.800079+0000","last_undegraded":"2024-09-16T04:57:59.800079+0000","last_fullsized":"2024-09-16T04:57:59.800079+0000","mapping_epoch":34,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5570560,"num_objects":1360,"num_object_clones":0,"num_object_copies":2720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15816,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'18945","reported_seq":19087,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.855861+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:57:59.855861+0000","last_peered":"2024-09-16T04:57:59.855861+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:57:59.855861+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5603328,"num_objects":1368,"num_object_clones":0,"num_object_copies":2736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15938,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2088}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18574","reported_seq":18733,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.771595+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:57:59.771595+0000","last_peered":"2024-09-16T04:57:59.771595+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:57:59.771595+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267456,"num_objects":1286,"num_object_clones":0,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15229,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2113}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18679","reported_seq":18905,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.848328+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:57:59.848328+0000","last_peered":"2024-09-16T04:57:59.848328+0000","last_clean":"2024-09-16T04:57:59.848328+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:57:59.848328+0000","last_undegraded":"2024-09-16T04:57:59.848328+0000","last_fullsized":"2024-09-16T04:57:59.848328+0000","mapping_epoch":34,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5476352,"num_objects":1337,"num_object_clones":0,"num_object_copies":2674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15749,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18799","reported_seq":19038,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.636391+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:57:59.636391+0000","last_peered":"2024-09-16T04:57:59.636391+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:57:59.636391+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5251072,"num_objects":1282,"num_object_clones":0,"num_object_copies":2564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15550,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2039}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18845","reported_seq":18983,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.797877+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:57:59.797877+0000","last_peered":"2024-09-16T04:57:59.797877+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:57:59.797877+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5623808,"num_objects":1373,"num_object_clones":0,"num_object_copies":2746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15500,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2092}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18576","reported_seq":25432,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.592586+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:57:59.592586+0000","last_peered":"2024-09-16T04:57:59.592586+0000","last_clean":"2024-09-16T04:57:59.592586+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:57:59.592586+0000","last_undegraded":"2024-09-16T04:57:59.592586+0000","last_fullsized":"2024-09-16T04:57:59.592586+0000","mapping_epoch":50,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:02.539106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5747922,"num_objects":1405,"num_object_clones":0,"num_object_copies":2810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18548,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18682","reported_seq":25549,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.617507+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:57:59.617507+0000","last_peered":"2024-09-16T04:57:59.617507+0000","last_clean":"2024-09-16T04:57:59.617507+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:57:59.617507+0000","last_undegraded":"2024-09-16T04:57:59.617507+0000","last_fullsized":"2024-09-16T04:57:59.617507+0000","mapping_epoch":46,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":1296,"num_object_clones":0,"num_object_copies":2592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15405,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18559","reported_seq":18665,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.665599+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:57:59.665599+0000","last_peered":"2024-09-16T04:57:59.665599+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:57:59.665599+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5395049,"num_objects":1318,"num_object_clones":0,"num_object_copies":2636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18558,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2089}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19159","reported_seq":19272,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.619781+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:57:59.619781+0000","last_peered":"2024-09-16T04:57:59.619781+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:57:59.619781+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5625659,"num_objects":1376,"num_object_clones":0,"num_object_copies":2752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19156,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2138}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18816","reported_seq":25620,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:57:59.348613+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:57:59.348613+0000","last_peered":"2024-09-16T04:57:59.348613+0000","last_clean":"2024-09-16T04:57:59.348613+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:57:59.348613+0000","last_undegraded":"2024-09-16T04:57:59.348613+0000","last_fullsized":"2024-09-16T04:57:59.348613+0000","mapping_epoch":48,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5660672,"num_objects":1382,"num_object_clones":0,"num_object_copies":2764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15570,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710453+0000","last_peered":"2024-09-16T04:56:42.710453+0000","last_clean":"2024-09-16T04:56:42.710453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710453+0000","last_undegraded":"2024-09-16T04:56:42.710453+0000","last_fullsized":"2024-09-16T04:56:42.710453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19078","reported_seq":19229,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:57:59.761140+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:57:59.761140+0000","last_peered":"2024-09-16T04:57:59.761140+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:57:59.761140+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5481065,"num_objects":1339,"num_object_clones":0,"num_object_copies":2678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15847,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2185}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":87830775,"num_objects":21450,"num_object_clones":0,"num_object_copies":42900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29372,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21450,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265180,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11370,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":342728704,"data_stored":342654459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161002,"ondisk_log_size":161002,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528788,"kb_used_data":136128,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93843052,"statfs":{"total":96636764160,"available":96095285248,"internally_reserved":0,"allocated":139395072,"data_stored":139101425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":402058921},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,7,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593829,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516044,"kb_used_data":65084,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93855796,"statfs":{"total":96636764160,"available":96108335104,"internally_reserved":0,"allocated":66646016,"data_stored":66296705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527792,"kb_used_data":135200,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93844048,"statfs":{"total":96636764160,"available":96096305152,"internally_reserved":0,"allocated":138444800,"data_stored":138155225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,0,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":137809920,"data_stored":137806441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66068480,"data_stored":66005265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":138850304,"data_stored":138842753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:06.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:58:06.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'18432", 'reported_seq': 21303, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:59.819604+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:57:59.819604+0000', 'last_peered': '2024-09-16T04:57:59.819604+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:57:59.819604+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'8400", 'ondisk_log_start': "47'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5349376, 'num_objects': 1306, 'num_object_clones': 0, 'num_object_copies': 2612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18432, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1391, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -645}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:06.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:58:06.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'18549", 'reported_seq': 18699, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:59.603964+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:57:59.603964+0000', 'last_peered': '2024-09-16T04:57:59.603964+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:57:59.603964+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'8500", 'ondisk_log_start': "47'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5357592, 'num_objects': 1309, 'num_object_clones': 0, 'num_object_copies': 2618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1309, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15288, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2010}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:06.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:58:06.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'18945", 'reported_seq': 19087, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:59.855861+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:57:59.855861+0000', 'last_peered': '2024-09-16T04:57:59.855861+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:57:59.855861+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'8900", 'ondisk_log_start': "47'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5603328, 'num_objects': 1368, 'num_object_clones': 0, 'num_object_copies': 2736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1368, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15938, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2088}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:06.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:58:06.085 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'18574", 'reported_seq': 18733, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:59.771595+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:57:59.771595+0000', 'last_peered': '2024-09-16T04:57:59.771595+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:57:59.771595+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'8500", 'ondisk_log_start': "47'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5267456, 'num_objects': 1286, 'num_object_clones': 0, 'num_object_copies': 2572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1286, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15229, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2113}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:06.086 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:58:06.086 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'18799", 'reported_seq': 19038, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:59.636391+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:57:59.636391+0000', 'last_peered': '2024-09-16T04:57:59.636391+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:57:59.636391+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'8700", 'ondisk_log_start': "47'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5251072, 'num_objects': 1282, 'num_object_clones': 0, 'num_object_copies': 2564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1282, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15550, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:06.086 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:58:06.086 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'18845", 'reported_seq': 18983, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:59.797877+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:57:59.797877+0000', 'last_peered': '2024-09-16T04:57:59.797877+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:57:59.797877+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'8800", 'ondisk_log_start': "47'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5623808, 'num_objects': 1373, 'num_object_clones': 0, 'num_object_copies': 2746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1373, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15500, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2092}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:06.086 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:58:06.086 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'18559", 'reported_seq': 18665, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:59.665599+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:57:59.665599+0000', 'last_peered': '2024-09-16T04:57:59.665599+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:57:59.665599+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'8500", 'ondisk_log_start': "47'8500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5395049, 'num_objects': 1318, 'num_object_clones': 0, 'num_object_copies': 2636, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1318, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18558, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2089}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:06.086 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:58:06.086 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'19159", 'reported_seq': 19272, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:59.619781+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:57:59.619781+0000', 'last_peered': '2024-09-16T04:57:59.619781+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:57:59.619781+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'9100", 'ondisk_log_start': "47'9100", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5625659, 'num_objects': 1376, 'num_object_clones': 0, 'num_object_copies': 2752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1376, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19156, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2138}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:06.086 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:58:06.086 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'19078", 'reported_seq': 19229, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:57:59.761140+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:57:59.761140+0000', 'last_peered': '2024-09-16T04:57:59.761140+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:57:59.761140+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'9000", 'ondisk_log_start': "47'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5481065, 'num_objects': 1339, 'num_object_clones': 0, 'num_object_copies': 2678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1339, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15847, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2185}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:06.087 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:58:07.180 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:07.180 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29272/38120 objects degraded (76.789%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":634,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":19060,"data_bytes":78033162,"bytes_used":1595097088,"bytes_avail":288315195392,"bytes_total":289910292480,"degraded_objects":29272,"degraded_total":38120,"degraded_ratio":0.76789087093389297,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":577},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:58:07.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:58:07.237 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:08.090 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:08.097 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:08.149 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-16T04:58:06.520738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":78033162,"num_objects":19060,"num_object_clones":0,"num_object_copies":38120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29272,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19060,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267574,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11471,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160697,"ondisk_log_size":160697,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1557712,"kb_used_data":317336,"kb_used_omap":12,"kb_used_meta":1240307,"kb_avail":281557808,"statfs":{"total":289910292480,"available":288315195392,"internally_reserved":0,"allocated":324952064,"data_stored":324019531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1270074369},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,5,11,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28409856,"num_objects":-6936,"num_object_clones":0,"num_object_copies":-13872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6936,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":285,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.015854"},"pg_stats":[{"pgid":"2.f","version":"51'18568","reported_seq":21639,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:04.842576+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:04.842576+0000","last_peered":"2024-09-16T04:58:04.842576+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:04.842576+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8500","ondisk_log_start":"47'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4792320,"num_objects":1170,"num_object_clones":0,"num_object_copies":2340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1851,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18568,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1491,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-681}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18835","reported_seq":18962,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:04.700412+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:04.700412+0000","last_peered":"2024-09-16T04:58:04.700412+0000","last_clean":"2024-09-16T04:58:04.700412+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:04.700412+0000","last_undegraded":"2024-09-16T04:58:04.700412+0000","last_fullsized":"2024-09-16T04:58:04.700412+0000","mapping_epoch":35,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4747264,"num_objects":1159,"num_object_clones":0,"num_object_copies":2318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15804,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18697","reported_seq":18847,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:04.610724+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:04.610724+0000","last_peered":"2024-09-16T04:58:04.610724+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:04.610724+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8600","ondisk_log_start":"47'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4751384,"num_objects":1161,"num_object_clones":0,"num_object_copies":2322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1161,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15436,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2158}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'19114","reported_seq":19194,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:04.632469+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:04.632469+0000","last_peered":"2024-09-16T04:58:04.632469+0000","last_clean":"2024-09-16T04:58:04.632469+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:04.632469+0000","last_undegraded":"2024-09-16T04:58:04.632469+0000","last_fullsized":"2024-09-16T04:58:04.632469+0000","mapping_epoch":35,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5038080,"num_objects":1230,"num_object_clones":0,"num_object_copies":2460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19114,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'18940","reported_seq":19169,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:04.776944+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:04.776944+0000","last_peered":"2024-09-16T04:58:04.776944+0000","last_clean":"2024-09-16T04:58:04.776944+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:04.776944+0000","last_undegraded":"2024-09-16T04:58:04.776944+0000","last_fullsized":"2024-09-16T04:58:04.776944+0000","mapping_epoch":34,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4931584,"num_objects":1204,"num_object_clones":0,"num_object_copies":2408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15972,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19095","reported_seq":19239,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:04.675723+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:04.675723+0000","last_peered":"2024-09-16T04:58:04.675723+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:04.675723+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4988928,"num_objects":1218,"num_object_clones":0,"num_object_copies":2436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16088,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2238}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18709","reported_seq":18868,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:04.677645+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:04.677645+0000","last_peered":"2024-09-16T04:58:04.677645+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:04.677645+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4714496,"num_objects":1151,"num_object_clones":0,"num_object_copies":2302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15364,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2248}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18841","reported_seq":19064,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:04.871206+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:04.871206+0000","last_peered":"2024-09-16T04:58:04.871206+0000","last_clean":"2024-09-16T04:58:04.871206+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:04.871206+0000","last_undegraded":"2024-09-16T04:58:04.871206+0000","last_fullsized":"2024-09-16T04:58:04.871206+0000","mapping_epoch":34,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4812800,"num_objects":1175,"num_object_clones":0,"num_object_copies":2350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15911,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'18942","reported_seq":19180,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:04.686276+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:04.686276+0000","last_peered":"2024-09-16T04:58:04.686276+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:04.686276+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4665344,"num_objects":1139,"num_object_clones":0,"num_object_copies":2278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15693,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2182}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'18997","reported_seq":19138,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:04.835861+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:04.835861+0000","last_peered":"2024-09-16T04:58:04.835861+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:04.835861+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5001216,"num_objects":1221,"num_object_clones":0,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15652,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2244}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18725","reported_seq":25581,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:04.677305+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:04.677305+0000","last_peered":"2024-09-16T04:58:04.677305+0000","last_clean":"2024-09-16T04:58:04.677305+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:04.677305+0000","last_undegraded":"2024-09-16T04:58:04.677305+0000","last_fullsized":"2024-09-16T04:58:04.677305+0000","mapping_epoch":50,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:07.626797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5137618,"num_objects":1256,"num_object_clones":0,"num_object_copies":2512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18697,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18813","reported_seq":25680,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:04.667298+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:04.667298+0000","last_peered":"2024-09-16T04:58:04.667298+0000","last_clean":"2024-09-16T04:58:04.667298+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:04.667298+0000","last_undegraded":"2024-09-16T04:58:04.667298+0000","last_fullsized":"2024-09-16T04:58:04.667298+0000","mapping_epoch":46,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4771840,"num_objects":1165,"num_object_clones":0,"num_object_copies":2330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15536,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18703","reported_seq":18811,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:04.573000+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:04.573000+0000","last_peered":"2024-09-16T04:58:04.573000+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:04.573000+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4805225,"num_objects":1174,"num_object_clones":0,"num_object_copies":2348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18702,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2233}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19318","reported_seq":19431,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:04.675872+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:04.675872+0000","last_peered":"2024-09-16T04:58:04.675872+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:04.675872+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4974395,"num_objects":1217,"num_object_clones":0,"num_object_copies":2434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19315,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2297}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'18983","reported_seq":25785,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:04.546002+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:04.546002+0000","last_peered":"2024-09-16T04:58:04.546002+0000","last_clean":"2024-09-16T04:58:04.546002+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:04.546002+0000","last_undegraded":"2024-09-16T04:58:04.546002+0000","last_fullsized":"2024-09-16T04:58:04.546002+0000","mapping_epoch":48,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4976640,"num_objects":1215,"num_object_clones":0,"num_object_copies":2430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15737,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19214","reported_seq":19365,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:04.815483+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:04.815483+0000","last_peered":"2024-09-16T04:58:04.815483+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:04.815483+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4924009,"num_objects":1203,"num_object_clones":0,"num_object_copies":2406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15983,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":78033143,"num_objects":19058,"num_object_clones":0,"num_object_copies":38116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29272,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19058,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267572,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11470,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":323014656,"data_stored":322940411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160694,"ondisk_log_size":160694,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531000,"kb_used_data":130024,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93840840,"statfs":{"total":96636764160,"available":96093020160,"internally_reserved":0,"allocated":133144576,"data_stored":132850929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":410578601},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593830,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500852,"kb_used_data":58208,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93870988,"statfs":{"total":96636764160,"available":96123891712,"internally_reserved":0,"allocated":59604992,"data_stored":59255681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525860,"kb_used_data":129104,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93845980,"statfs":{"total":96636764160,"available":96098283520,"internally_reserved":0,"allocated":132202496,"data_stored":131912921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,5,0,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":131551232,"data_stored":131547753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58789888,"data_stored":58726673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":132673536,"data_stored":132665985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:11.152 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:11.958 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:11.959 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:11.996 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-16T04:58:10.521672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":68428042,"num_objects":16715,"num_object_clones":0,"num_object_copies":33430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269919,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11559,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160942,"ondisk_log_size":160942,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536252,"kb_used_data":300040,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281579268,"statfs":{"total":289910292480,"available":288337170432,"internally_reserved":0,"allocated":307240960,"data_stored":306308427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1265814529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,9,10,10,15,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29167616,"num_objects":-7121,"num_object_clones":0,"num_object_copies":-14242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-292,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7121,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.010670"},"pg_stats":[{"pgid":"2.f","version":"51'18729","reported_seq":21975,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.814368+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:09.814368+0000","last_peered":"2024-09-16T04:58:09.814368+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:09.814368+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4132864,"num_objects":1009,"num_object_clones":0,"num_object_copies":2018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1763,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18729,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1579,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-754}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18988","reported_seq":19114,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.657499+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:09.657499+0000","last_peered":"2024-09-16T04:58:09.657499+0000","last_clean":"2024-09-16T04:58:09.657499+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:09.657499+0000","last_undegraded":"2024-09-16T04:58:09.657499+0000","last_fullsized":"2024-09-16T04:58:09.657499+0000","mapping_epoch":35,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4120576,"num_objects":1006,"num_object_clones":0,"num_object_copies":2012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15957,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18831","reported_seq":18981,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.675944+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:09.675944+0000","last_peered":"2024-09-16T04:58:09.675944+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:09.675944+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4202520,"num_objects":1027,"num_object_clones":0,"num_object_copies":2054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1027,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15570,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2292}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'19259","reported_seq":19338,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.701207+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:09.701207+0000","last_peered":"2024-09-16T04:58:09.701207+0000","last_clean":"2024-09-16T04:58:09.701207+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:09.701207+0000","last_undegraded":"2024-09-16T04:58:09.701207+0000","last_fullsized":"2024-09-16T04:58:09.701207+0000","mapping_epoch":35,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4444160,"num_objects":1085,"num_object_clones":0,"num_object_copies":2170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19259,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19093","reported_seq":19322,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.831599+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:09.831599+0000","last_peered":"2024-09-16T04:58:09.831599+0000","last_clean":"2024-09-16T04:58:09.831599+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:09.831599+0000","last_undegraded":"2024-09-16T04:58:09.831599+0000","last_fullsized":"2024-09-16T04:58:09.831599+0000","mapping_epoch":34,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304896,"num_objects":1051,"num_object_clones":0,"num_object_copies":2102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16125,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19237","reported_seq":19381,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.713464+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:09.713464+0000","last_peered":"2024-09-16T04:58:09.713464+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:09.713464+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4407296,"num_objects":1076,"num_object_clones":0,"num_object_copies":2152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16230,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2380}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18862","reported_seq":19021,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.858675+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:09.858675+0000","last_peered":"2024-09-16T04:58:09.858675+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:09.858675+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4087808,"num_objects":998,"num_object_clones":0,"num_object_copies":1996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15517,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2401}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18985","reported_seq":19212,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.896210+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:09.896210+0000","last_peered":"2024-09-16T04:58:09.896210+0000","last_clean":"2024-09-16T04:58:09.896210+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:09.896210+0000","last_undegraded":"2024-09-16T04:58:09.896210+0000","last_fullsized":"2024-09-16T04:58:09.896210+0000","mapping_epoch":34,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4222976,"num_objects":1031,"num_object_clones":0,"num_object_copies":2062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16055,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19077","reported_seq":19316,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.525851+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:09.525851+0000","last_peered":"2024-09-16T04:58:09.525851+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:09.525851+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4112384,"num_objects":1004,"num_object_clones":0,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15828,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2317}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19151","reported_seq":19292,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.822233+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:09.822233+0000","last_peered":"2024-09-16T04:58:09.822233+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:09.822233+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4370432,"num_objects":1067,"num_object_clones":0,"num_object_copies":2134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2398}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18862","reported_seq":25718,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.737751+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:09.737751+0000","last_peered":"2024-09-16T04:58:09.737751+0000","last_clean":"2024-09-16T04:58:09.737751+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:09.737751+0000","last_undegraded":"2024-09-16T04:58:09.737751+0000","last_fullsized":"2024-09-16T04:58:09.737751+0000","mapping_epoch":50,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:12.745048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4576466,"num_objects":1119,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18834,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18951","reported_seq":25818,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.737546+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:09.737546+0000","last_peered":"2024-09-16T04:58:09.737546+0000","last_clean":"2024-09-16T04:58:09.737546+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:09.737546+0000","last_undegraded":"2024-09-16T04:58:09.737546+0000","last_fullsized":"2024-09-16T04:58:09.737546+0000","mapping_epoch":46,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4206592,"num_objects":1027,"num_object_clones":0,"num_object_copies":2054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15674,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18853","reported_seq":18959,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.672260+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:09.672260+0000","last_peered":"2024-09-16T04:58:09.672260+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:09.672260+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190825,"num_objects":1024,"num_object_clones":0,"num_object_copies":2048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18852,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2383}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19483","reported_seq":19594,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.725165+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:09.725165+0000","last_peered":"2024-09-16T04:58:09.725165+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:09.725165+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4298555,"num_objects":1052,"num_object_clones":0,"num_object_copies":2104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19480,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2462}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'19124","reported_seq":25928,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.580971+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:09.580971+0000","last_peered":"2024-09-16T04:58:09.580971+0000","last_clean":"2024-09-16T04:58:09.580971+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:09.580971+0000","last_undegraded":"2024-09-16T04:58:09.580971+0000","last_fullsized":"2024-09-16T04:58:09.580971+0000","mapping_epoch":48,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4399104,"num_objects":1074,"num_object_clones":0,"num_object_copies":2148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15878,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19354","reported_seq":19505,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.825776+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:09.825776+0000","last_peered":"2024-09-16T04:58:09.825776+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:09.825776+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4350569,"num_objects":1063,"num_object_clones":0,"num_object_copies":2126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16123,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2461}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":68428023,"num_objects":16713,"num_object_clones":0,"num_object_copies":33426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16713,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269917,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11558,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":303788032,"data_stored":303713787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160939,"ondisk_log_size":160939,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513912,"kb_used_data":125412,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93857928,"statfs":{"total":96636764160,"available":96110518272,"internally_reserved":0,"allocated":128421888,"data_stored":128128241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":397799081},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,5,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593831,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498188,"kb_used_data":51388,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93873652,"statfs":{"total":96636764160,"available":96126619648,"internally_reserved":0,"allocated":52621312,"data_stored":52272001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,5,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524152,"kb_used_data":123240,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93847688,"statfs":{"total":96636764160,"available":96100032512,"internally_reserved":0,"allocated":126197760,"data_stored":125908185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,1,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":125480960,"data_stored":125477481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51630080,"data_stored":51566865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126676992,"data_stored":126669441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:11.998 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:13.012 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:13.013 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:13.117 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-16T04:58:12.522150+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":68428042,"num_objects":16715,"num_object_clones":0,"num_object_copies":33430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269919,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11559,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160942,"ondisk_log_size":160942,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536252,"kb_used_data":300040,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281579268,"statfs":{"total":289910292480,"available":288337170432,"internally_reserved":0,"allocated":307240960,"data_stored":306308427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1265814529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,9,10,10,15,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19402752,"num_objects":-4737,"num_object_clones":0,"num_object_copies":-9474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4737,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009084"},"pg_stats":[{"pgid":"2.f","version":"51'18729","reported_seq":21975,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.814368+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:09.814368+0000","last_peered":"2024-09-16T04:58:09.814368+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:09.814368+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4132864,"num_objects":1009,"num_object_clones":0,"num_object_copies":2018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1763,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18729,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1579,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-754}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18988","reported_seq":19114,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.657499+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:09.657499+0000","last_peered":"2024-09-16T04:58:09.657499+0000","last_clean":"2024-09-16T04:58:09.657499+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:09.657499+0000","last_undegraded":"2024-09-16T04:58:09.657499+0000","last_fullsized":"2024-09-16T04:58:09.657499+0000","mapping_epoch":35,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4120576,"num_objects":1006,"num_object_clones":0,"num_object_copies":2012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15957,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18831","reported_seq":18981,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.675944+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:09.675944+0000","last_peered":"2024-09-16T04:58:09.675944+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:09.675944+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4202520,"num_objects":1027,"num_object_clones":0,"num_object_copies":2054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1027,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15570,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2292}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'19259","reported_seq":19338,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.701207+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:09.701207+0000","last_peered":"2024-09-16T04:58:09.701207+0000","last_clean":"2024-09-16T04:58:09.701207+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:09.701207+0000","last_undegraded":"2024-09-16T04:58:09.701207+0000","last_fullsized":"2024-09-16T04:58:09.701207+0000","mapping_epoch":35,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4444160,"num_objects":1085,"num_object_clones":0,"num_object_copies":2170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19259,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19093","reported_seq":19322,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.831599+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:09.831599+0000","last_peered":"2024-09-16T04:58:09.831599+0000","last_clean":"2024-09-16T04:58:09.831599+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:09.831599+0000","last_undegraded":"2024-09-16T04:58:09.831599+0000","last_fullsized":"2024-09-16T04:58:09.831599+0000","mapping_epoch":34,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304896,"num_objects":1051,"num_object_clones":0,"num_object_copies":2102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16125,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19237","reported_seq":19381,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.713464+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:09.713464+0000","last_peered":"2024-09-16T04:58:09.713464+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:09.713464+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4407296,"num_objects":1076,"num_object_clones":0,"num_object_copies":2152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16230,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2380}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18862","reported_seq":19021,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.858675+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:09.858675+0000","last_peered":"2024-09-16T04:58:09.858675+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:09.858675+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4087808,"num_objects":998,"num_object_clones":0,"num_object_copies":1996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15517,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2401}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18985","reported_seq":19212,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.896210+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:09.896210+0000","last_peered":"2024-09-16T04:58:09.896210+0000","last_clean":"2024-09-16T04:58:09.896210+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:09.896210+0000","last_undegraded":"2024-09-16T04:58:09.896210+0000","last_fullsized":"2024-09-16T04:58:09.896210+0000","mapping_epoch":34,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4222976,"num_objects":1031,"num_object_clones":0,"num_object_copies":2062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16055,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19077","reported_seq":19316,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.525851+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:09.525851+0000","last_peered":"2024-09-16T04:58:09.525851+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:09.525851+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4112384,"num_objects":1004,"num_object_clones":0,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15828,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2317}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19151","reported_seq":19292,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.822233+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:09.822233+0000","last_peered":"2024-09-16T04:58:09.822233+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:09.822233+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4370432,"num_objects":1067,"num_object_clones":0,"num_object_copies":2134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2398}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18862","reported_seq":25718,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.737751+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:09.737751+0000","last_peered":"2024-09-16T04:58:09.737751+0000","last_clean":"2024-09-16T04:58:09.737751+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:09.737751+0000","last_undegraded":"2024-09-16T04:58:09.737751+0000","last_fullsized":"2024-09-16T04:58:09.737751+0000","mapping_epoch":50,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:12.745048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4576466,"num_objects":1119,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18834,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18951","reported_seq":25818,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.737546+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:09.737546+0000","last_peered":"2024-09-16T04:58:09.737546+0000","last_clean":"2024-09-16T04:58:09.737546+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:09.737546+0000","last_undegraded":"2024-09-16T04:58:09.737546+0000","last_fullsized":"2024-09-16T04:58:09.737546+0000","mapping_epoch":46,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4206592,"num_objects":1027,"num_object_clones":0,"num_object_copies":2054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15674,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18853","reported_seq":18959,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.672260+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:09.672260+0000","last_peered":"2024-09-16T04:58:09.672260+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:09.672260+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190825,"num_objects":1024,"num_object_clones":0,"num_object_copies":2048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18852,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2383}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19483","reported_seq":19594,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.725165+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:09.725165+0000","last_peered":"2024-09-16T04:58:09.725165+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:09.725165+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4298555,"num_objects":1052,"num_object_clones":0,"num_object_copies":2104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19480,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2462}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'19124","reported_seq":25928,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.580971+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:09.580971+0000","last_peered":"2024-09-16T04:58:09.580971+0000","last_clean":"2024-09-16T04:58:09.580971+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:09.580971+0000","last_undegraded":"2024-09-16T04:58:09.580971+0000","last_fullsized":"2024-09-16T04:58:09.580971+0000","mapping_epoch":48,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4399104,"num_objects":1074,"num_object_clones":0,"num_object_copies":2148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15878,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19354","reported_seq":19505,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.825776+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:09.825776+0000","last_peered":"2024-09-16T04:58:09.825776+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:09.825776+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4350569,"num_objects":1063,"num_object_clones":0,"num_object_copies":2126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16123,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2461}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":68428023,"num_objects":16713,"num_object_clones":0,"num_object_copies":33426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16713,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269917,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11558,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":303788032,"data_stored":303713787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160939,"ondisk_log_size":160939,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513912,"kb_used_data":125412,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93857928,"statfs":{"total":96636764160,"available":96110518272,"internally_reserved":0,"allocated":128421888,"data_stored":128128241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":397799081},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,5,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593831,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498188,"kb_used_data":51388,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93873652,"statfs":{"total":96636764160,"available":96126619648,"internally_reserved":0,"allocated":52621312,"data_stored":52272001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,5,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524152,"kb_used_data":123240,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93847688,"statfs":{"total":96636764160,"available":96100032512,"internally_reserved":0,"allocated":126197760,"data_stored":125908185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,1,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":125480960,"data_stored":125477481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51630080,"data_stored":51566865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126676992,"data_stored":126669441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:13.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:58:13.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'18729", 'reported_seq': 21975, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:09.814368+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:58:09.814368+0000', 'last_peered': '2024-09-16T04:58:09.814368+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:58:09.814368+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'8700", 'ondisk_log_start': "47'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4132864, 'num_objects': 1009, 'num_object_clones': 0, 'num_object_copies': 2018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1763, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1009, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18729, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1579, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -754}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:13.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:58:13.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'18831", 'reported_seq': 18981, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:09.675944+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:09.675944+0000', 'last_peered': '2024-09-16T04:58:09.675944+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:09.675944+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'8800", 'ondisk_log_start': "47'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4202520, 'num_objects': 1027, 'num_object_clones': 0, 'num_object_copies': 2054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1027, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15570, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2292}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:13.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:58:13.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'19237", 'reported_seq': 19381, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:09.713464+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:09.713464+0000', 'last_peered': '2024-09-16T04:58:09.713464+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:09.713464+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'9200", 'ondisk_log_start': "47'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4407296, 'num_objects': 1076, 'num_object_clones': 0, 'num_object_copies': 2152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1076, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16230, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2380}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:13.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:58:13.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'18862", 'reported_seq': 19021, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:09.858675+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:09.858675+0000', 'last_peered': '2024-09-16T04:58:09.858675+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:09.858675+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'8800", 'ondisk_log_start': "47'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4087808, 'num_objects': 998, 'num_object_clones': 0, 'num_object_copies': 1996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 998, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15517, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2401}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:13.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:58:13.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'19077", 'reported_seq': 19316, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:09.525851+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:09.525851+0000', 'last_peered': '2024-09-16T04:58:09.525851+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:09.525851+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'9000", 'ondisk_log_start': "47'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4112384, 'num_objects': 1004, 'num_object_clones': 0, 'num_object_copies': 2008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15828, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2317}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:13.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:58:13.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'19151", 'reported_seq': 19292, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:09.822233+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:09.822233+0000', 'last_peered': '2024-09-16T04:58:09.822233+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:09.822233+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'9100", 'ondisk_log_start': "47'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4370432, 'num_objects': 1067, 'num_object_clones': 0, 'num_object_copies': 2134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1067, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15806, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2398}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:13.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:58:13.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'18853", 'reported_seq': 18959, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:09.672260+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:09.672260+0000', 'last_peered': '2024-09-16T04:58:09.672260+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:09.672260+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'8800", 'ondisk_log_start': "47'8800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4190825, 'num_objects': 1024, 'num_object_clones': 0, 'num_object_copies': 2048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18852, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2383}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:13.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:58:13.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'19483", 'reported_seq': 19594, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:09.725165+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:58:09.725165+0000', 'last_peered': '2024-09-16T04:58:09.725165+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:58:09.725165+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'9400", 'ondisk_log_start': "47'9400", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4298555, 'num_objects': 1052, 'num_object_clones': 0, 'num_object_copies': 2104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1052, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19480, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2462}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:13.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:58:13.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'19354", 'reported_seq': 19505, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:09.825776+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:09.825776+0000', 'last_peered': '2024-09-16T04:58:09.825776+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:09.825776+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'9300", 'ondisk_log_start': "47'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4350569, 'num_objects': 1063, 'num_object_clones': 0, 'num_object_copies': 2126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1063, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16123, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2461}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:13.121 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:58:14.230 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:14.230 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29184/33430 objects degraded (87.299%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":642,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":16715,"data_bytes":68428042,"bytes_used":1573122048,"bytes_avail":288337170432,"bytes_total":289910292480,"degraded_objects":29184,"degraded_total":33430,"degraded_ratio":0.87298833383188745,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":188,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":394},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:58:14.263 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:58:14.264 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:15.091 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:15.091 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:15.161 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-16T04:58:14.522606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":68428042,"num_objects":16715,"num_object_clones":0,"num_object_copies":33430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269919,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11559,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160942,"ondisk_log_size":160942,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536252,"kb_used_data":300040,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281579268,"statfs":{"total":289910292480,"available":288337170432,"internally_reserved":0,"allocated":307240960,"data_stored":306308427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1265814529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,9,10,10,15,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19402752,"num_objects":-4737,"num_object_clones":0,"num_object_copies":-9474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4737,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004626"},"pg_stats":[{"pgid":"2.f","version":"51'18729","reported_seq":21975,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.814368+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:09.814368+0000","last_peered":"2024-09-16T04:58:09.814368+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:09.814368+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8700","ondisk_log_start":"47'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4132864,"num_objects":1009,"num_object_clones":0,"num_object_copies":2018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1763,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18729,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1579,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-754}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'18988","reported_seq":19114,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.657499+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:09.657499+0000","last_peered":"2024-09-16T04:58:09.657499+0000","last_clean":"2024-09-16T04:58:09.657499+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:09.657499+0000","last_undegraded":"2024-09-16T04:58:09.657499+0000","last_fullsized":"2024-09-16T04:58:09.657499+0000","mapping_epoch":35,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4120576,"num_objects":1006,"num_object_clones":0,"num_object_copies":2012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15957,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18831","reported_seq":18981,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.675944+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:09.675944+0000","last_peered":"2024-09-16T04:58:09.675944+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:09.675944+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4202520,"num_objects":1027,"num_object_clones":0,"num_object_copies":2054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1027,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15570,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2292}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'19259","reported_seq":19338,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.701207+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:09.701207+0000","last_peered":"2024-09-16T04:58:09.701207+0000","last_clean":"2024-09-16T04:58:09.701207+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:09.701207+0000","last_undegraded":"2024-09-16T04:58:09.701207+0000","last_fullsized":"2024-09-16T04:58:09.701207+0000","mapping_epoch":35,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4444160,"num_objects":1085,"num_object_clones":0,"num_object_copies":2170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19259,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19093","reported_seq":19322,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.831599+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:09.831599+0000","last_peered":"2024-09-16T04:58:09.831599+0000","last_clean":"2024-09-16T04:58:09.831599+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:09.831599+0000","last_undegraded":"2024-09-16T04:58:09.831599+0000","last_fullsized":"2024-09-16T04:58:09.831599+0000","mapping_epoch":34,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304896,"num_objects":1051,"num_object_clones":0,"num_object_copies":2102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16125,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19237","reported_seq":19381,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.713464+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:09.713464+0000","last_peered":"2024-09-16T04:58:09.713464+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:09.713464+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4407296,"num_objects":1076,"num_object_clones":0,"num_object_copies":2152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16230,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2380}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'18862","reported_seq":19021,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.858675+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:09.858675+0000","last_peered":"2024-09-16T04:58:09.858675+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:09.858675+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4087808,"num_objects":998,"num_object_clones":0,"num_object_copies":1996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15517,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2401}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'18985","reported_seq":19212,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.896210+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:09.896210+0000","last_peered":"2024-09-16T04:58:09.896210+0000","last_clean":"2024-09-16T04:58:09.896210+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:09.896210+0000","last_undegraded":"2024-09-16T04:58:09.896210+0000","last_fullsized":"2024-09-16T04:58:09.896210+0000","mapping_epoch":34,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4222976,"num_objects":1031,"num_object_clones":0,"num_object_copies":2062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16055,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19077","reported_seq":19316,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.525851+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:09.525851+0000","last_peered":"2024-09-16T04:58:09.525851+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:09.525851+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4112384,"num_objects":1004,"num_object_clones":0,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15828,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2317}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19151","reported_seq":19292,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.822233+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:09.822233+0000","last_peered":"2024-09-16T04:58:09.822233+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:09.822233+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4370432,"num_objects":1067,"num_object_clones":0,"num_object_copies":2134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15806,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2398}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'18862","reported_seq":25718,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.737751+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:09.737751+0000","last_peered":"2024-09-16T04:58:09.737751+0000","last_clean":"2024-09-16T04:58:09.737751+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:09.737751+0000","last_undegraded":"2024-09-16T04:58:09.737751+0000","last_fullsized":"2024-09-16T04:58:09.737751+0000","mapping_epoch":50,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:12.745048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4576466,"num_objects":1119,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18834,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'18951","reported_seq":25818,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.737546+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:09.737546+0000","last_peered":"2024-09-16T04:58:09.737546+0000","last_clean":"2024-09-16T04:58:09.737546+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:09.737546+0000","last_undegraded":"2024-09-16T04:58:09.737546+0000","last_fullsized":"2024-09-16T04:58:09.737546+0000","mapping_epoch":46,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4206592,"num_objects":1027,"num_object_clones":0,"num_object_copies":2054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15674,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18853","reported_seq":18959,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.672260+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:09.672260+0000","last_peered":"2024-09-16T04:58:09.672260+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:09.672260+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190825,"num_objects":1024,"num_object_clones":0,"num_object_copies":2048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18852,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2383}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19483","reported_seq":19594,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.725165+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:09.725165+0000","last_peered":"2024-09-16T04:58:09.725165+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:09.725165+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4298555,"num_objects":1052,"num_object_clones":0,"num_object_copies":2104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19480,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2462}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'19124","reported_seq":25928,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:09.580971+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:09.580971+0000","last_peered":"2024-09-16T04:58:09.580971+0000","last_clean":"2024-09-16T04:58:09.580971+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:09.580971+0000","last_undegraded":"2024-09-16T04:58:09.580971+0000","last_fullsized":"2024-09-16T04:58:09.580971+0000","mapping_epoch":48,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4399104,"num_objects":1074,"num_object_clones":0,"num_object_copies":2148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15878,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19354","reported_seq":19505,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:09.825776+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:09.825776+0000","last_peered":"2024-09-16T04:58:09.825776+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:09.825776+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4350569,"num_objects":1063,"num_object_clones":0,"num_object_copies":2126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16123,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2461}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":68428023,"num_objects":16713,"num_object_clones":0,"num_object_copies":33426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16713,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269917,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11558,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":303788032,"data_stored":303713787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160939,"ondisk_log_size":160939,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513912,"kb_used_data":125412,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93857928,"statfs":{"total":96636764160,"available":96110518272,"internally_reserved":0,"allocated":128421888,"data_stored":128128241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":397799081},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,5,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593831,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498188,"kb_used_data":51388,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93873652,"statfs":{"total":96636764160,"available":96126619648,"internally_reserved":0,"allocated":52621312,"data_stored":52272001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,5,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524152,"kb_used_data":123240,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93847688,"statfs":{"total":96636764160,"available":96100032512,"internally_reserved":0,"allocated":126197760,"data_stored":125908185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,1,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":125480960,"data_stored":125477481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51630080,"data_stored":51566865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126676992,"data_stored":126669441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:18.163 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:19.115 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:19.116 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:19.182 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-16T04:58:18.524860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":58773770,"num_objects":14358,"num_object_clones":0,"num_object_copies":28716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272276,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11661,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1516348,"kb_used_data":280140,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281599172,"statfs":{"total":289910292480,"available":288357552128,"internally_reserved":0,"allocated":286863360,"data_stored":285930827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1265814529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,8,6,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19259392,"num_objects":-4702,"num_object_clones":0,"num_object_copies":-9404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4702,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004122"},"pg_stats":[{"pgid":"2.f","version":"51'18878","reported_seq":22329,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.830905+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:14.830905+0000","last_peered":"2024-09-16T04:58:14.830905+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:14.830905+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3522560,"num_objects":860,"num_object_clones":0,"num_object_copies":1720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18878,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1681,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-801}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19128","reported_seq":19255,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.779654+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:14.779654+0000","last_peered":"2024-09-16T04:58:14.779654+0000","last_clean":"2024-09-16T04:58:14.779654+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:14.779654+0000","last_undegraded":"2024-09-16T04:58:14.779654+0000","last_fullsized":"2024-09-16T04:58:14.779654+0000","mapping_epoch":35,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3547136,"num_objects":866,"num_object_clones":0,"num_object_copies":1732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16097,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18976","reported_seq":19126,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.675820+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:14.675820+0000","last_peered":"2024-09-16T04:58:14.675820+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:14.675820+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3608600,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15715,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2437}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'19418","reported_seq":19499,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.736931+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:14.736931+0000","last_peered":"2024-09-16T04:58:14.736931+0000","last_clean":"2024-09-16T04:58:14.736931+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:14.736931+0000","last_undegraded":"2024-09-16T04:58:14.736931+0000","last_fullsized":"2024-09-16T04:58:14.736931+0000","mapping_epoch":35,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3792896,"num_objects":926,"num_object_clones":0,"num_object_copies":1852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19418,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19246","reported_seq":19475,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.790574+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:14.790574+0000","last_peered":"2024-09-16T04:58:14.790574+0000","last_clean":"2024-09-16T04:58:14.790574+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:14.790574+0000","last_undegraded":"2024-09-16T04:58:14.790574+0000","last_fullsized":"2024-09-16T04:58:14.790574+0000","mapping_epoch":34,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3678208,"num_objects":898,"num_object_clones":0,"num_object_copies":1796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19389","reported_seq":19533,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.879216+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:14.879216+0000","last_peered":"2024-09-16T04:58:14.879216+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:14.879216+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3784704,"num_objects":924,"num_object_clones":0,"num_object_copies":1848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16382,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19005","reported_seq":19164,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.826254+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:14.826254+0000","last_peered":"2024-09-16T04:58:14.826254+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:14.826254+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502080,"num_objects":855,"num_object_clones":0,"num_object_copies":1710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15660,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19117","reported_seq":19344,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.845926+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:14.845926+0000","last_peered":"2024-09-16T04:58:14.845926+0000","last_clean":"2024-09-16T04:58:14.845926+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:14.845926+0000","last_undegraded":"2024-09-16T04:58:14.845926+0000","last_fullsized":"2024-09-16T04:58:14.845926+0000","mapping_epoch":34,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3682304,"num_objects":899,"num_object_clones":0,"num_object_copies":1798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16187,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19200","reported_seq":19439,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.806948+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:14.806948+0000","last_peered":"2024-09-16T04:58:14.806948+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:14.806948+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3608576,"num_objects":881,"num_object_clones":0,"num_object_copies":1762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15951,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2440}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19306","reported_seq":19447,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.882160+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:14.882160+0000","last_peered":"2024-09-16T04:58:14.882160+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:14.882160+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":912,"num_object_clones":0,"num_object_copies":1824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15961,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2553}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19043","reported_seq":25898,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.790486+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:14.790486+0000","last_peered":"2024-09-16T04:58:14.790486+0000","last_clean":"2024-09-16T04:58:14.790486+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:14.790486+0000","last_undegraded":"2024-09-16T04:58:14.790486+0000","last_fullsized":"2024-09-16T04:58:14.790486+0000","mapping_epoch":50,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:18.748025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3835090,"num_objects":938,"num_object_clones":0,"num_object_copies":1876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19015,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19074","reported_seq":25941,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.722959+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:14.722959+0000","last_peered":"2024-09-16T04:58:14.722959+0000","last_clean":"2024-09-16T04:58:14.722959+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:14.722959+0000","last_undegraded":"2024-09-16T04:58:14.722959+0000","last_fullsized":"2024-09-16T04:58:14.722959+0000","mapping_epoch":46,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3702784,"num_objects":904,"num_object_clones":0,"num_object_copies":1808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15797,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18997","reported_seq":19105,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.746528+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:14.746528+0000","last_peered":"2024-09-16T04:58:14.746528+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:14.746528+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3601001,"num_objects":880,"num_object_clones":0,"num_object_copies":1760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18996,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2527}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19623","reported_seq":19737,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.798698+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:14.798698+0000","last_peered":"2024-09-16T04:58:14.798698+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:14.798698+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3725115,"num_objects":912,"num_object_clones":0,"num_object_copies":1824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19620,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2602}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'19289","reported_seq":26093,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.289041+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:14.289041+0000","last_peered":"2024-09-16T04:58:14.289041+0000","last_clean":"2024-09-16T04:58:14.289041+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:14.289041+0000","last_undegraded":"2024-09-16T04:58:14.289041+0000","last_fullsized":"2024-09-16T04:58:14.289041+0000","mapping_epoch":48,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3723264,"num_objects":909,"num_object_clones":0,"num_object_copies":1818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16043,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19507","reported_seq":19658,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.859290+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:14.859290+0000","last_peered":"2024-09-16T04:58:14.859290+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:14.859290+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3723881,"num_objects":910,"num_object_clones":0,"num_object_copies":1820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16276,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":58773751,"num_objects":14356,"num_object_clones":0,"num_object_copies":28712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272274,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11660,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":284442624,"data_stored":284368379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511080,"kb_used_data":118424,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93860760,"statfs":{"total":96636764160,"available":96113418240,"internally_reserved":0,"allocated":121266176,"data_stored":120972529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":402058921},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593832,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499416,"kb_used_data":44296,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93872424,"statfs":{"total":96636764160,"available":96125362176,"internally_reserved":0,"allocated":45359104,"data_stored":45009793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":466022062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505852,"kb_used_data":117420,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93865988,"statfs":{"total":96636764160,"available":96118771712,"internally_reserved":0,"allocated":120238080,"data_stored":119948505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":119361536,"data_stored":119358057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44355584,"data_stored":44292369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":120725504,"data_stored":120717953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:19.184 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:20.121 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:20.123 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:20.178 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-16T04:58:18.524860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":58773770,"num_objects":14358,"num_object_clones":0,"num_object_copies":28716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272276,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11661,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1516348,"kb_used_data":280140,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281599172,"statfs":{"total":289910292480,"available":288357552128,"internally_reserved":0,"allocated":286863360,"data_stored":285930827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1265814529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,8,6,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19259392,"num_objects":-4702,"num_object_clones":0,"num_object_copies":-9404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4702,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004122"},"pg_stats":[{"pgid":"2.f","version":"51'18878","reported_seq":22329,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.830905+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:14.830905+0000","last_peered":"2024-09-16T04:58:14.830905+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:14.830905+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8800","ondisk_log_start":"47'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3522560,"num_objects":860,"num_object_clones":0,"num_object_copies":1720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1661,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18878,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1681,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-801}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19128","reported_seq":19255,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.779654+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:14.779654+0000","last_peered":"2024-09-16T04:58:14.779654+0000","last_clean":"2024-09-16T04:58:14.779654+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:14.779654+0000","last_undegraded":"2024-09-16T04:58:14.779654+0000","last_fullsized":"2024-09-16T04:58:14.779654+0000","mapping_epoch":35,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3547136,"num_objects":866,"num_object_clones":0,"num_object_copies":1732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16097,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'18976","reported_seq":19126,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.675820+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:14.675820+0000","last_peered":"2024-09-16T04:58:14.675820+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:14.675820+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3608600,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15715,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2437}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'19418","reported_seq":19499,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.736931+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:14.736931+0000","last_peered":"2024-09-16T04:58:14.736931+0000","last_clean":"2024-09-16T04:58:14.736931+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:14.736931+0000","last_undegraded":"2024-09-16T04:58:14.736931+0000","last_fullsized":"2024-09-16T04:58:14.736931+0000","mapping_epoch":35,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3792896,"num_objects":926,"num_object_clones":0,"num_object_copies":1852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19418,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19246","reported_seq":19475,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.790574+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:14.790574+0000","last_peered":"2024-09-16T04:58:14.790574+0000","last_clean":"2024-09-16T04:58:14.790574+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:14.790574+0000","last_undegraded":"2024-09-16T04:58:14.790574+0000","last_fullsized":"2024-09-16T04:58:14.790574+0000","mapping_epoch":34,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3678208,"num_objects":898,"num_object_clones":0,"num_object_copies":1796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19389","reported_seq":19533,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.879216+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:14.879216+0000","last_peered":"2024-09-16T04:58:14.879216+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:14.879216+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3784704,"num_objects":924,"num_object_clones":0,"num_object_copies":1848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16382,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19005","reported_seq":19164,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.826254+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:14.826254+0000","last_peered":"2024-09-16T04:58:14.826254+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:14.826254+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502080,"num_objects":855,"num_object_clones":0,"num_object_copies":1710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15660,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19117","reported_seq":19344,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.845926+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:14.845926+0000","last_peered":"2024-09-16T04:58:14.845926+0000","last_clean":"2024-09-16T04:58:14.845926+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:14.845926+0000","last_undegraded":"2024-09-16T04:58:14.845926+0000","last_fullsized":"2024-09-16T04:58:14.845926+0000","mapping_epoch":34,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3682304,"num_objects":899,"num_object_clones":0,"num_object_copies":1798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16187,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19200","reported_seq":19439,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.806948+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:14.806948+0000","last_peered":"2024-09-16T04:58:14.806948+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:14.806948+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3608576,"num_objects":881,"num_object_clones":0,"num_object_copies":1762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15951,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2440}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19306","reported_seq":19447,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.882160+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:14.882160+0000","last_peered":"2024-09-16T04:58:14.882160+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:14.882160+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":912,"num_object_clones":0,"num_object_copies":1824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15961,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2553}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19043","reported_seq":25898,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.790486+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:14.790486+0000","last_peered":"2024-09-16T04:58:14.790486+0000","last_clean":"2024-09-16T04:58:14.790486+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:14.790486+0000","last_undegraded":"2024-09-16T04:58:14.790486+0000","last_fullsized":"2024-09-16T04:58:14.790486+0000","mapping_epoch":50,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:18.748025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3835090,"num_objects":938,"num_object_clones":0,"num_object_copies":1876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19015,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19074","reported_seq":25941,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.722959+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:14.722959+0000","last_peered":"2024-09-16T04:58:14.722959+0000","last_clean":"2024-09-16T04:58:14.722959+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:14.722959+0000","last_undegraded":"2024-09-16T04:58:14.722959+0000","last_fullsized":"2024-09-16T04:58:14.722959+0000","mapping_epoch":46,"log_start":"47'9000","ondisk_log_start":"47'9000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3702784,"num_objects":904,"num_object_clones":0,"num_object_copies":1808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15797,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'18997","reported_seq":19105,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.746528+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:14.746528+0000","last_peered":"2024-09-16T04:58:14.746528+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:14.746528+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3601001,"num_objects":880,"num_object_clones":0,"num_object_copies":1760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18996,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2527}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19623","reported_seq":19737,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.798698+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:14.798698+0000","last_peered":"2024-09-16T04:58:14.798698+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:14.798698+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3725115,"num_objects":912,"num_object_clones":0,"num_object_copies":1824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19620,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2602}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'19289","reported_seq":26093,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:14.289041+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:14.289041+0000","last_peered":"2024-09-16T04:58:14.289041+0000","last_clean":"2024-09-16T04:58:14.289041+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:14.289041+0000","last_undegraded":"2024-09-16T04:58:14.289041+0000","last_fullsized":"2024-09-16T04:58:14.289041+0000","mapping_epoch":48,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3723264,"num_objects":909,"num_object_clones":0,"num_object_copies":1818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16043,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710892+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710892+0000","last_peered":"2024-09-16T04:56:42.710892+0000","last_clean":"2024-09-16T04:56:42.710892+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710892+0000","last_undegraded":"2024-09-16T04:56:42.710892+0000","last_fullsized":"2024-09-16T04:56:42.710892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19507","reported_seq":19658,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:14.859290+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:14.859290+0000","last_peered":"2024-09-16T04:58:14.859290+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:14.859290+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3723881,"num_objects":910,"num_object_clones":0,"num_object_copies":1820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16276,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":58773751,"num_objects":14356,"num_object_clones":0,"num_object_copies":28712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272274,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11660,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":284442624,"data_stored":284368379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511080,"kb_used_data":118424,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93860760,"statfs":{"total":96636764160,"available":96113418240,"internally_reserved":0,"allocated":121266176,"data_stored":120972529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":402058921},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593832,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":499416,"kb_used_data":44296,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93872424,"statfs":{"total":96636764160,"available":96125362176,"internally_reserved":0,"allocated":45359104,"data_stored":45009793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":466022062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505852,"kb_used_data":117420,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93865988,"statfs":{"total":96636764160,"available":96118771712,"internally_reserved":0,"allocated":120238080,"data_stored":119948505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":397733546},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":119361536,"data_stored":119358057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44355584,"data_stored":44292369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":120725504,"data_stored":120717953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:20.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:58:20.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'18878", 'reported_seq': 22329, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:14.830905+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:58:14.830905+0000', 'last_peered': '2024-09-16T04:58:14.830905+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:58:14.830905+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'8800", 'ondisk_log_start': "47'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3522560, 'num_objects': 860, 'num_object_clones': 0, 'num_object_copies': 1720, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1661, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 860, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18878, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1681, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -801}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:20.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:58:20.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'18976", 'reported_seq': 19126, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:14.675820+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:14.675820+0000', 'last_peered': '2024-09-16T04:58:14.675820+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:14.675820+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'8900", 'ondisk_log_start': "47'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3608600, 'num_objects': 882, 'num_object_clones': 0, 'num_object_copies': 1764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 882, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15715, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2437}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:20.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:58:20.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'19389", 'reported_seq': 19533, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:14.879216+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:14.879216+0000', 'last_peered': '2024-09-16T04:58:14.879216+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:14.879216+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'9300", 'ondisk_log_start': "47'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3784704, 'num_objects': 924, 'num_object_clones': 0, 'num_object_copies': 1848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 924, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16382, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:20.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:58:20.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19005", 'reported_seq': 19164, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:14.826254+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:14.826254+0000', 'last_peered': '2024-09-16T04:58:14.826254+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:14.826254+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9000", 'ondisk_log_start': "47'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3502080, 'num_objects': 855, 'num_object_clones': 0, 'num_object_copies': 1710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 855, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15660, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:20.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:58:20.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'19200", 'reported_seq': 19439, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:14.806948+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:14.806948+0000', 'last_peered': '2024-09-16T04:58:14.806948+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:14.806948+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'9100", 'ondisk_log_start': "47'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3608576, 'num_objects': 881, 'num_object_clones': 0, 'num_object_copies': 1762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 881, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15951, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2440}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:20.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:58:20.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'19306", 'reported_seq': 19447, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:14.882160+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:14.882160+0000', 'last_peered': '2024-09-16T04:58:14.882160+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:14.882160+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'9300", 'ondisk_log_start': "47'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3735552, 'num_objects': 912, 'num_object_clones': 0, 'num_object_copies': 1824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 912, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15961, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2553}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:20.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:58:20.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'18997", 'reported_seq': 19105, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:14.746528+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:14.746528+0000', 'last_peered': '2024-09-16T04:58:14.746528+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:14.746528+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'8900", 'ondisk_log_start': "47'8900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3601001, 'num_objects': 880, 'num_object_clones': 0, 'num_object_copies': 1760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 880, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18996, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2527}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:20.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:58:20.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'19623", 'reported_seq': 19737, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:14.798698+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:58:14.798698+0000', 'last_peered': '2024-09-16T04:58:14.798698+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:58:14.798698+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'9600", 'ondisk_log_start': "47'9600", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3725115, 'num_objects': 912, 'num_object_clones': 0, 'num_object_copies': 1824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 912, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19620, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2602}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:20.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:58:20.181 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'19507", 'reported_seq': 19658, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:14.859290+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:14.859290+0000', 'last_peered': '2024-09-16T04:58:14.859290+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:14.859290+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'9500", 'ondisk_log_start': "47'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3723881, 'num_objects': 910, 'num_object_clones': 0, 'num_object_copies': 1820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 910, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16276, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:20.181 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:58:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:21.280 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28964/23896 objects degraded (121.209%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":649,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":11948,"data_bytes":48902410,"bytes_used":1560137728,"bytes_avail":288350154752,"bytes_total":289910292480,"degraded_objects":28964,"degraded_total":23896,"degraded_ratio":1.2120857047204554,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":308,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":592},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:58:21.354 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:58:21.354 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:22.148 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:22.150 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:22.199 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-09-16T04:58:20.528787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48902410,"num_objects":11948,"num_object_clones":0,"num_object_copies":23896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11948,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11779,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160909,"ondisk_log_size":160909,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523572,"kb_used_data":260988,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281591948,"statfs":{"total":289910292480,"available":288350154752,"internally_reserved":0,"allocated":267251712,"data_stored":266319179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,5,4,14,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29130752,"num_objects":-7112,"num_object_clones":0,"num_object_copies":-14224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-308,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7112,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007547"},"pg_stats":[{"pgid":"2.f","version":"51'18999","reported_seq":22685,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.897366+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:19.897366+0000","last_peered":"2024-09-16T04:58:19.897366+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:19.897366+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3026944,"num_objects":739,"num_object_clones":0,"num_object_copies":1478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18999,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1799,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-804}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19272","reported_seq":19399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.830854+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:19.830854+0000","last_peered":"2024-09-16T04:58:19.830854+0000","last_clean":"2024-09-16T04:58:19.830854+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:19.830854+0000","last_undegraded":"2024-09-16T04:58:19.830854+0000","last_fullsized":"2024-09-16T04:58:19.830854+0000","mapping_epoch":35,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2957312,"num_objects":722,"num_object_clones":0,"num_object_copies":1444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16241,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19108","reported_seq":19256,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.861147+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:19.861147+0000","last_peered":"2024-09-16T04:58:19.861147+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:19.861147+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3067928,"num_objects":750,"num_object_clones":0,"num_object_copies":1500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":750,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15847,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2569}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'19566","reported_seq":19647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.792922+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:19.792922+0000","last_peered":"2024-09-16T04:58:19.792922+0000","last_clean":"2024-09-16T04:58:19.792922+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:19.792922+0000","last_undegraded":"2024-09-16T04:58:19.792922+0000","last_fullsized":"2024-09-16T04:58:19.792922+0000","mapping_epoch":35,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3186688,"num_objects":778,"num_object_clones":0,"num_object_copies":1556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19566,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19390","reported_seq":19619,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.822469+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:19.822469+0000","last_peered":"2024-09-16T04:58:19.822469+0000","last_clean":"2024-09-16T04:58:19.822469+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:19.822469+0000","last_undegraded":"2024-09-16T04:58:19.822469+0000","last_fullsized":"2024-09-16T04:58:19.822469+0000","mapping_epoch":34,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3088384,"num_objects":754,"num_object_clones":0,"num_object_copies":1508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16422,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19547","reported_seq":19689,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.881268+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:19.881268+0000","last_peered":"2024-09-16T04:58:19.881268+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:19.881268+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3137536,"num_objects":766,"num_object_clones":0,"num_object_copies":1532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2690}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19141","reported_seq":19300,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.822626+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:19.822626+0000","last_peered":"2024-09-16T04:58:19.822626+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:19.822626+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2945024,"num_objects":719,"num_object_clones":0,"num_object_copies":1438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15796,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2680}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19294","reported_seq":19521,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.862099+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:19.862099+0000","last_peered":"2024-09-16T04:58:19.862099+0000","last_clean":"2024-09-16T04:58:19.862099+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:19.862099+0000","last_undegraded":"2024-09-16T04:58:19.862099+0000","last_fullsized":"2024-09-16T04:58:19.862099+0000","mapping_epoch":34,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2957312,"num_objects":722,"num_object_clones":0,"num_object_copies":1444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16364,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19355","reported_seq":19594,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.758085+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:19.758085+0000","last_peered":"2024-09-16T04:58:19.758085+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:19.758085+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2973696,"num_objects":726,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16106,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2595}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19462","reported_seq":19603,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.838771+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:19.838771+0000","last_peered":"2024-09-16T04:58:19.838771+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:19.838771+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3096576,"num_objects":756,"num_object_clones":0,"num_object_copies":1512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16117,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2709}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19213","reported_seq":26069,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.759695+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:19.759695+0000","last_peered":"2024-09-16T04:58:19.759695+0000","last_clean":"2024-09-16T04:58:19.759695+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:19.759695+0000","last_undegraded":"2024-09-16T04:58:19.759695+0000","last_fullsized":"2024-09-16T04:58:19.759695+0000","mapping_epoch":50,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:23.804510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3138770,"num_objects":768,"num_object_clones":0,"num_object_copies":1536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19185,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19231","reported_seq":26098,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.736846+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:19.736846+0000","last_peered":"2024-09-16T04:58:19.736846+0000","last_clean":"2024-09-16T04:58:19.736846+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:19.736846+0000","last_undegraded":"2024-09-16T04:58:19.736846+0000","last_fullsized":"2024-09-16T04:58:19.736846+0000","mapping_epoch":46,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3059712,"num_objects":747,"num_object_clones":0,"num_object_copies":1494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15954,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19150","reported_seq":19255,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.792924+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:19.792924+0000","last_peered":"2024-09-16T04:58:19.792924+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:19.792924+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2974313,"num_objects":727,"num_object_clones":0,"num_object_copies":1454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19149,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2680}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19787","reported_seq":19901,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.792436+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:19.792436+0000","last_peered":"2024-09-16T04:58:19.792436+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:19.792436+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3053371,"num_objects":748,"num_object_clones":0,"num_object_copies":1496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19784,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2766}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'19421","reported_seq":26226,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.660052+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:19.660052+0000","last_peered":"2024-09-16T04:58:19.660052+0000","last_clean":"2024-09-16T04:58:19.660052+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:19.660052+0000","last_undegraded":"2024-09-16T04:58:19.660052+0000","last_fullsized":"2024-09-16T04:58:19.660052+0000","mapping_epoch":48,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3182592,"num_objects":777,"num_object_clones":0,"num_object_copies":1554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16175,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19670","reported_seq":19821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.836905+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:19.836905+0000","last_peered":"2024-09-16T04:58:19.836905+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:19.836905+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3056233,"num_objects":747,"num_object_clones":0,"num_object_copies":1494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16439,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2777}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":48902391,"num_objects":11946,"num_object_clones":0,"num_object_copies":23892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274684,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11778,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":264851456,"data_stored":264777211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160906,"ondisk_log_size":160906,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513228,"kb_used_data":112248,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93858612,"statfs":{"total":96636764160,"available":96111218688,"internally_reserved":0,"allocated":114941952,"data_stored":114648305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":410578601},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593833,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483956,"kb_used_data":37156,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93887884,"statfs":{"total":96636764160,"available":96141193216,"internally_reserved":0,"allocated":38047744,"data_stored":37698433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526388,"kb_used_data":111584,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93845452,"statfs":{"total":96636764160,"available":96097742848,"internally_reserved":0,"allocated":114262016,"data_stored":113972441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":113348608,"data_stored":113345129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37052416,"data_stored":36989201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114450432,"data_stored":114442881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:25.202 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:26.639 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:26.640 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-09-16T04:58:24.533241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48902410,"num_objects":11948,"num_object_clones":0,"num_object_copies":23896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11948,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274686,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11779,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160909,"ondisk_log_size":160909,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1523572,"kb_used_data":260988,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281591948,"statfs":{"total":289910292480,"available":288350154752,"internally_reserved":0,"allocated":267251712,"data_stored":266319179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1292815361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,5,4,14,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19525632,"num_objects":-4767,"num_object_clones":0,"num_object_copies":-9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-220,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4767,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011090"},"pg_stats":[{"pgid":"2.f","version":"51'18999","reported_seq":22685,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.897366+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:19.897366+0000","last_peered":"2024-09-16T04:58:19.897366+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:19.897366+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'8900","ondisk_log_start":"47'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3026944,"num_objects":739,"num_object_clones":0,"num_object_copies":1478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18999,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1799,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-804}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19272","reported_seq":19399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.830854+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:19.830854+0000","last_peered":"2024-09-16T04:58:19.830854+0000","last_clean":"2024-09-16T04:58:19.830854+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:19.830854+0000","last_undegraded":"2024-09-16T04:58:19.830854+0000","last_fullsized":"2024-09-16T04:58:19.830854+0000","mapping_epoch":35,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2957312,"num_objects":722,"num_object_clones":0,"num_object_copies":1444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16241,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19108","reported_seq":19256,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.861147+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:19.861147+0000","last_peered":"2024-09-16T04:58:19.861147+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:19.861147+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3067928,"num_objects":750,"num_object_clones":0,"num_object_copies":1500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":750,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15847,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2569}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'19566","reported_seq":19647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.792922+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:19.792922+0000","last_peered":"2024-09-16T04:58:19.792922+0000","last_clean":"2024-09-16T04:58:19.792922+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:19.792922+0000","last_undegraded":"2024-09-16T04:58:19.792922+0000","last_fullsized":"2024-09-16T04:58:19.792922+0000","mapping_epoch":35,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3186688,"num_objects":778,"num_object_clones":0,"num_object_copies":1556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19566,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19390","reported_seq":19619,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.822469+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:19.822469+0000","last_peered":"2024-09-16T04:58:19.822469+0000","last_clean":"2024-09-16T04:58:19.822469+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:19.822469+0000","last_undegraded":"2024-09-16T04:58:19.822469+0000","last_fullsized":"2024-09-16T04:58:19.822469+0000","mapping_epoch":34,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3088384,"num_objects":754,"num_object_clones":0,"num_object_copies":1508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16422,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19547","reported_seq":19689,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.881268+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:19.881268+0000","last_peered":"2024-09-16T04:58:19.881268+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:19.881268+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3137536,"num_objects":766,"num_object_clones":0,"num_object_copies":1532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2690}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19141","reported_seq":19300,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.822626+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:19.822626+0000","last_peered":"2024-09-16T04:58:19.822626+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:19.822626+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2945024,"num_objects":719,"num_object_clones":0,"num_object_copies":1438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15796,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2680}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19294","reported_seq":19521,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.862099+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:19.862099+0000","last_peered":"2024-09-16T04:58:19.862099+0000","last_clean":"2024-09-16T04:58:19.862099+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:19.862099+0000","last_undegraded":"2024-09-16T04:58:19.862099+0000","last_fullsized":"2024-09-16T04:58:19.862099+0000","mapping_epoch":34,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2957312,"num_objects":722,"num_object_clones":0,"num_object_copies":1444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16364,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19355","reported_seq":19594,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.758085+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:19.758085+0000","last_peered":"2024-09-16T04:58:19.758085+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:19.758085+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2973696,"num_objects":726,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16106,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2595}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19462","reported_seq":19603,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.838771+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:19.838771+0000","last_peered":"2024-09-16T04:58:19.838771+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:19.838771+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3096576,"num_objects":756,"num_object_clones":0,"num_object_copies":1512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16117,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2709}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19213","reported_seq":26069,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.759695+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:19.759695+0000","last_peered":"2024-09-16T04:58:19.759695+0000","last_clean":"2024-09-16T04:58:19.759695+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:19.759695+0000","last_undegraded":"2024-09-16T04:58:19.759695+0000","last_fullsized":"2024-09-16T04:58:19.759695+0000","mapping_epoch":50,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:23.804510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3138770,"num_objects":768,"num_object_clones":0,"num_object_copies":1536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19185,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19231","reported_seq":26098,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.736846+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:19.736846+0000","last_peered":"2024-09-16T04:58:19.736846+0000","last_clean":"2024-09-16T04:58:19.736846+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:19.736846+0000","last_undegraded":"2024-09-16T04:58:19.736846+0000","last_fullsized":"2024-09-16T04:58:19.736846+0000","mapping_epoch":46,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3059712,"num_objects":747,"num_object_clones":0,"num_object_copies":1494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15954,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19150","reported_seq":19255,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.792924+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:19.792924+0000","last_peered":"2024-09-16T04:58:19.792924+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:19.792924+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2974313,"num_objects":727,"num_object_clones":0,"num_object_copies":1454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19149,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2680}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19787","reported_seq":19901,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.792436+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:19.792436+0000","last_peered":"2024-09-16T04:58:19.792436+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:19.792436+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3053371,"num_objects":748,"num_object_clones":0,"num_object_copies":1496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19784,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2766}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'19421","reported_seq":26226,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:19.660052+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:19.660052+0000","last_peered":"2024-09-16T04:58:19.660052+0000","last_clean":"2024-09-16T04:58:19.660052+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:19.660052+0000","last_undegraded":"2024-09-16T04:58:19.660052+0000","last_fullsized":"2024-09-16T04:58:19.660052+0000","mapping_epoch":48,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3182592,"num_objects":777,"num_object_clones":0,"num_object_copies":1554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16175,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19670","reported_seq":19821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:19.836905+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:19.836905+0000","last_peered":"2024-09-16T04:58:19.836905+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:19.836905+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3056233,"num_objects":747,"num_object_clones":0,"num_object_copies":1494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16439,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2777}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":48902391,"num_objects":11946,"num_object_clones":0,"num_object_copies":23892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11946,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274684,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11778,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":264851456,"data_stored":264777211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160906,"ondisk_log_size":160906,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513228,"kb_used_data":112248,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93858612,"statfs":{"total":96636764160,"available":96111218688,"internally_reserved":0,"allocated":114941952,"data_stored":114648305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":410578601},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593833,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483956,"kb_used_data":37156,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93887884,"statfs":{"total":96636764160,"available":96141193216,"internally_reserved":0,"allocated":38047744,"data_stored":37698433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526388,"kb_used_data":111584,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93845452,"statfs":{"total":96636764160,"available":96097742848,"internally_reserved":0,"allocated":114262016,"data_stored":113972441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":113348608,"data_stored":113345129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37052416,"data_stored":36989201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114450432,"data_stored":114442881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:26.642 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:27.603 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:27.606 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:27.651 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-09-16T04:58:26.533651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39428362,"num_objects":9635,"num_object_clones":0,"num_object_copies":19270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9635,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":276999,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11897,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160922,"ondisk_log_size":160922,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501048,"kb_used_data":242608,"kb_used_omap":12,"kb_used_meta":1258355,"kb_avail":281614472,"statfs":{"total":289910292480,"available":288373219328,"internally_reserved":0,"allocated":248430592,"data_stored":247498059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1288555521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,5,5,10,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28999680,"num_objects":-7080,"num_object_clones":0,"num_object_copies":-14160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7080,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011045"},"pg_stats":[{"pgid":"2.f","version":"51'19155","reported_seq":23078,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.936066+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:24.936066+0000","last_peered":"2024-09-16T04:58:24.936066+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:24.936066+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2387968,"num_objects":583,"num_object_clones":0,"num_object_copies":1166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1425,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19155,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1917,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-842}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19382","reported_seq":19509,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.813156+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:24.813156+0000","last_peered":"2024-09-16T04:58:24.813156+0000","last_clean":"2024-09-16T04:58:24.813156+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:24.813156+0000","last_undegraded":"2024-09-16T04:58:24.813156+0000","last_fullsized":"2024-09-16T04:58:24.813156+0000","mapping_epoch":35,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2506752,"num_objects":612,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16351,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19276","reported_seq":19426,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.785098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:24.785098+0000","last_peered":"2024-09-16T04:58:24.785098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:24.785098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379800,"num_objects":582,"num_object_clones":0,"num_object_copies":1164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":582,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16015,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2737}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'19711","reported_seq":19792,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.663412+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:24.663412+0000","last_peered":"2024-09-16T04:58:24.663412+0000","last_clean":"2024-09-16T04:58:24.663412+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:24.663412+0000","last_undegraded":"2024-09-16T04:58:24.663412+0000","last_fullsized":"2024-09-16T04:58:24.663412+0000","mapping_epoch":35,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2592768,"num_objects":633,"num_object_clones":0,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19711,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19524","reported_seq":19752,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.861435+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:24.861435+0000","last_peered":"2024-09-16T04:58:24.861435+0000","last_clean":"2024-09-16T04:58:24.861435+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:24.861435+0000","last_undegraded":"2024-09-16T04:58:24.861435+0000","last_fullsized":"2024-09-16T04:58:24.861435+0000","mapping_epoch":34,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2539520,"num_objects":620,"num_object_clones":0,"num_object_copies":1240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16556,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19700","reported_seq":19844,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.894277+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:24.894277+0000","last_peered":"2024-09-16T04:58:24.894277+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:24.894277+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2510848,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16693,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2843}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19294","reported_seq":19453,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.841876+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:24.841876+0000","last_peered":"2024-09-16T04:58:24.841876+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:24.841876+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2318336,"num_objects":566,"num_object_clones":0,"num_object_copies":1132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15949,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19420","reported_seq":19646,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.936614+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:24.936614+0000","last_peered":"2024-09-16T04:58:24.936614+0000","last_clean":"2024-09-16T04:58:24.936614+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:24.936614+0000","last_undegraded":"2024-09-16T04:58:24.936614+0000","last_fullsized":"2024-09-16T04:58:24.936614+0000","mapping_epoch":34,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2441216,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16490,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19485","reported_seq":19724,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.789846+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:24.789846+0000","last_peered":"2024-09-16T04:58:24.789846+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:24.789846+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2441216,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16236,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2725}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19626","reported_seq":19767,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.895343+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:24.895343+0000","last_peered":"2024-09-16T04:58:24.895343+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:24.895343+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16281,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19366","reported_seq":26222,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.837429+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:24.837429+0000","last_peered":"2024-09-16T04:58:24.837429+0000","last_clean":"2024-09-16T04:58:24.837429+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:24.837429+0000","last_undegraded":"2024-09-16T04:58:24.837429+0000","last_fullsized":"2024-09-16T04:58:24.837429+0000","mapping_epoch":50,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:28.827597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2512082,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19338,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19380","reported_seq":26247,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.824814+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:24.824814+0000","last_peered":"2024-09-16T04:58:24.824814+0000","last_clean":"2024-09-16T04:58:24.824814+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:24.824814+0000","last_undegraded":"2024-09-16T04:58:24.824814+0000","last_fullsized":"2024-09-16T04:58:24.824814+0000","mapping_epoch":46,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2449408,"num_objects":598,"num_object_clones":0,"num_object_copies":1196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16103,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19278","reported_seq":19387,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.818021+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:24.818021+0000","last_peered":"2024-09-16T04:58:24.818021+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:24.818021+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2450025,"num_objects":599,"num_object_clones":0,"num_object_copies":1198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19277,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2808}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19942","reported_seq":20056,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.800108+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:24.800108+0000","last_peered":"2024-09-16T04:58:24.800108+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:24.800108+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2418491,"num_objects":593,"num_object_clones":0,"num_object_copies":1186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19939,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2921}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'19569","reported_seq":26374,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.710440+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:24.710440+0000","last_peered":"2024-09-16T04:58:24.710440+0000","last_clean":"2024-09-16T04:58:24.710440+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:24.710440+0000","last_undegraded":"2024-09-16T04:58:24.710440+0000","last_fullsized":"2024-09-16T04:58:24.710440+0000","mapping_epoch":48,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2576384,"num_objects":629,"num_object_clones":0,"num_object_copies":1258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16323,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19811","reported_seq":19962,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.881389+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:24.881389+0000","last_peered":"2024-09-16T04:58:24.881389+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:24.881389+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2478697,"num_objects":606,"num_object_clones":0,"num_object_copies":1212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16580,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":39428343,"num_objects":9633,"num_object_clones":0,"num_object_copies":19266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":276997,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11896,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245788672,"data_stored":245714427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160919,"ondisk_log_size":160919,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495176,"kb_used_data":106660,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93876664,"statfs":{"total":96636764160,"available":96129703936,"internally_reserved":0,"allocated":109219840,"data_stored":108926193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":397799081},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,6,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593834,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481032,"kb_used_data":30068,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93890808,"statfs":{"total":96636764160,"available":96144187392,"internally_reserved":0,"allocated":30789632,"data_stored":30440321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524840,"kb_used_data":105880,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93847000,"statfs":{"total":96636764160,"available":96099328000,"internally_reserved":0,"allocated":108421120,"data_stored":108131545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,0,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":107466752,"data_stored":107463273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29605888,"data_stored":29542673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":108716032,"data_stored":108708481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:27.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:58:27.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'19155", 'reported_seq': 23078, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:24.936066+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:58:24.936066+0000', 'last_peered': '2024-09-16T04:58:24.936066+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:58:24.936066+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'9100", 'ondisk_log_start': "47'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2387968, 'num_objects': 583, 'num_object_clones': 0, 'num_object_copies': 1166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1425, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 583, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19155, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1917, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -842}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:27.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:58:27.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19276", 'reported_seq': 19426, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:24.785098+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:24.785098+0000', 'last_peered': '2024-09-16T04:58:24.785098+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:24.785098+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9200", 'ondisk_log_start': "47'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2379800, 'num_objects': 582, 'num_object_clones': 0, 'num_object_copies': 1164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 582, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16015, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2737}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:27.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:58:27.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'19700", 'reported_seq': 19844, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:24.894277+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:24.894277+0000', 'last_peered': '2024-09-16T04:58:24.894277+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:24.894277+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'9600", 'ondisk_log_start': "47'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2510848, 'num_objects': 613, 'num_object_clones': 0, 'num_object_copies': 1226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 613, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16693, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2843}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:27.653 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:58:27.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19294", 'reported_seq': 19453, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:24.841876+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:24.841876+0000', 'last_peered': '2024-09-16T04:58:24.841876+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:24.841876+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9200", 'ondisk_log_start': "47'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2318336, 'num_objects': 566, 'num_object_clones': 0, 'num_object_copies': 1132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15949, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2833}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:27.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:58:27.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'19485", 'reported_seq': 19724, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:24.789846+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:24.789846+0000', 'last_peered': '2024-09-16T04:58:24.789846+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:24.789846+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'9400", 'ondisk_log_start': "47'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2441216, 'num_objects': 596, 'num_object_clones': 0, 'num_object_copies': 1192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 596, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16236, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2725}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:27.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:58:27.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'19626", 'reported_seq': 19767, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:24.895343+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:24.895343+0000', 'last_peered': '2024-09-16T04:58:24.895343+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:24.895343+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'9600", 'ondisk_log_start': "47'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2424832, 'num_objects': 592, 'num_object_clones': 0, 'num_object_copies': 1184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 592, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16281, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2873}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:27.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:58:27.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19278", 'reported_seq': 19387, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:24.818021+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:24.818021+0000', 'last_peered': '2024-09-16T04:58:24.818021+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:24.818021+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9200", 'ondisk_log_start': "47'9200", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2450025, 'num_objects': 599, 'num_object_clones': 0, 'num_object_copies': 1198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 599, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19277, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2808}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:27.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:58:27.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'19942", 'reported_seq': 20056, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:24.800108+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:58:24.800108+0000', 'last_peered': '2024-09-16T04:58:24.800108+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:58:24.800108+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'9900", 'ondisk_log_start': "47'9900", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2418491, 'num_objects': 593, 'num_object_clones': 0, 'num_object_copies': 1186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 593, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19939, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2921}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:27.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:58:27.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'19811", 'reported_seq': 19962, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:24.881389+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:24.881389+0000', 'last_peered': '2024-09-16T04:58:24.881389+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:24.881389+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2478697, 'num_objects': 606, 'num_object_clones': 0, 'num_object_copies': 1212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 606, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16580, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2918}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:27.655 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:58:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:28.601 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28846/19270 objects degraded (149.694%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":656,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9635,"data_bytes":39428362,"bytes_used":1537073152,"bytes_avail":288373219328,"bytes_total":289910292480,"degraded_objects":28846,"degraded_total":19270,"degraded_ratio":1.4969382459782046,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":338,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":589},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:58:28.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:58:28.656 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:29.321 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:29.416 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-16T04:58:28.534127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39428362,"num_objects":9635,"num_object_clones":0,"num_object_copies":19270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9635,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":276999,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11897,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160922,"ondisk_log_size":160922,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501048,"kb_used_data":242608,"kb_used_omap":12,"kb_used_meta":1258355,"kb_avail":281614472,"statfs":{"total":289910292480,"available":288373219328,"internally_reserved":0,"allocated":248430592,"data_stored":247498059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1288555521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,5,5,10,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19345408,"num_objects":-4723,"num_object_clones":0,"num_object_copies":-9446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4723,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011070"},"pg_stats":[{"pgid":"2.f","version":"51'19155","reported_seq":23078,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.936066+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:24.936066+0000","last_peered":"2024-09-16T04:58:24.936066+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:24.936066+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9100","ondisk_log_start":"47'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2387968,"num_objects":583,"num_object_clones":0,"num_object_copies":1166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1425,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19155,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1917,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-842}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19382","reported_seq":19509,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.813156+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:24.813156+0000","last_peered":"2024-09-16T04:58:24.813156+0000","last_clean":"2024-09-16T04:58:24.813156+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:24.813156+0000","last_undegraded":"2024-09-16T04:58:24.813156+0000","last_fullsized":"2024-09-16T04:58:24.813156+0000","mapping_epoch":35,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2506752,"num_objects":612,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16351,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19276","reported_seq":19426,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.785098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:24.785098+0000","last_peered":"2024-09-16T04:58:24.785098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:24.785098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379800,"num_objects":582,"num_object_clones":0,"num_object_copies":1164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":582,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16015,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2737}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'19711","reported_seq":19792,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.663412+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:24.663412+0000","last_peered":"2024-09-16T04:58:24.663412+0000","last_clean":"2024-09-16T04:58:24.663412+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:24.663412+0000","last_undegraded":"2024-09-16T04:58:24.663412+0000","last_fullsized":"2024-09-16T04:58:24.663412+0000","mapping_epoch":35,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2592768,"num_objects":633,"num_object_clones":0,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19711,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19524","reported_seq":19752,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.861435+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:24.861435+0000","last_peered":"2024-09-16T04:58:24.861435+0000","last_clean":"2024-09-16T04:58:24.861435+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:24.861435+0000","last_undegraded":"2024-09-16T04:58:24.861435+0000","last_fullsized":"2024-09-16T04:58:24.861435+0000","mapping_epoch":34,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2539520,"num_objects":620,"num_object_clones":0,"num_object_copies":1240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16556,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19700","reported_seq":19844,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.894277+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:24.894277+0000","last_peered":"2024-09-16T04:58:24.894277+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:24.894277+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2510848,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16693,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2843}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19294","reported_seq":19453,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.841876+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:24.841876+0000","last_peered":"2024-09-16T04:58:24.841876+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:24.841876+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2318336,"num_objects":566,"num_object_clones":0,"num_object_copies":1132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15949,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2833}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19420","reported_seq":19646,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.936614+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:24.936614+0000","last_peered":"2024-09-16T04:58:24.936614+0000","last_clean":"2024-09-16T04:58:24.936614+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:24.936614+0000","last_undegraded":"2024-09-16T04:58:24.936614+0000","last_fullsized":"2024-09-16T04:58:24.936614+0000","mapping_epoch":34,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2441216,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16490,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19485","reported_seq":19724,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.789846+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:24.789846+0000","last_peered":"2024-09-16T04:58:24.789846+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:24.789846+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2441216,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16236,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2725}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19626","reported_seq":19767,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.895343+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:24.895343+0000","last_peered":"2024-09-16T04:58:24.895343+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:24.895343+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16281,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19366","reported_seq":26222,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.837429+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:24.837429+0000","last_peered":"2024-09-16T04:58:24.837429+0000","last_clean":"2024-09-16T04:58:24.837429+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:24.837429+0000","last_undegraded":"2024-09-16T04:58:24.837429+0000","last_fullsized":"2024-09-16T04:58:24.837429+0000","mapping_epoch":50,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:28.827597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2512082,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19338,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19380","reported_seq":26247,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.824814+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:24.824814+0000","last_peered":"2024-09-16T04:58:24.824814+0000","last_clean":"2024-09-16T04:58:24.824814+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:24.824814+0000","last_undegraded":"2024-09-16T04:58:24.824814+0000","last_fullsized":"2024-09-16T04:58:24.824814+0000","mapping_epoch":46,"log_start":"47'9300","ondisk_log_start":"47'9300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2449408,"num_objects":598,"num_object_clones":0,"num_object_copies":1196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16103,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19278","reported_seq":19387,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.818021+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:24.818021+0000","last_peered":"2024-09-16T04:58:24.818021+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:24.818021+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2450025,"num_objects":599,"num_object_clones":0,"num_object_copies":1198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19277,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2808}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'19942","reported_seq":20056,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.800108+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:24.800108+0000","last_peered":"2024-09-16T04:58:24.800108+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:24.800108+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2418491,"num_objects":593,"num_object_clones":0,"num_object_copies":1186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19939,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2921}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'19569","reported_seq":26374,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:24.710440+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:24.710440+0000","last_peered":"2024-09-16T04:58:24.710440+0000","last_clean":"2024-09-16T04:58:24.710440+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:24.710440+0000","last_undegraded":"2024-09-16T04:58:24.710440+0000","last_fullsized":"2024-09-16T04:58:24.710440+0000","mapping_epoch":48,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2576384,"num_objects":629,"num_object_clones":0,"num_object_copies":1258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16323,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19811","reported_seq":19962,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:24.881389+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:24.881389+0000","last_peered":"2024-09-16T04:58:24.881389+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:24.881389+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2478697,"num_objects":606,"num_object_clones":0,"num_object_copies":1212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16580,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":39428343,"num_objects":9633,"num_object_clones":0,"num_object_copies":19266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":276997,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11896,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245788672,"data_stored":245714427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160919,"ondisk_log_size":160919,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495176,"kb_used_data":106660,"kb_used_omap":4,"kb_used_meta":388475,"kb_avail":93876664,"statfs":{"total":96636764160,"available":96129703936,"internally_reserved":0,"allocated":109219840,"data_stored":108926193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":397799081},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,6,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593834,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481032,"kb_used_data":30068,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93890808,"statfs":{"total":96636764160,"available":96144187392,"internally_reserved":0,"allocated":30789632,"data_stored":30440321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":461762222},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524840,"kb_used_data":105880,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93847000,"statfs":{"total":96636764160,"available":96099328000,"internally_reserved":0,"allocated":108421120,"data_stored":108131545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,0,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":107466752,"data_stored":107463273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29605888,"data_stored":29542673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":108716032,"data_stored":108708481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:32.418 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:33.261 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:33.262 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:33.344 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-16T04:58:32.536686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29868298,"num_objects":7301,"num_object_clones":0,"num_object_copies":14602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279333,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12009,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160756,"ondisk_log_size":160756,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483972,"kb_used_data":224136,"kb_used_omap":12,"kb_used_meta":1259763,"kb_avail":281631548,"statfs":{"total":289910292480,"available":288390705152,"internally_reserved":0,"allocated":229515264,"data_stored":228582731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1289997313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,7,3,8,7,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19034112,"num_objects":-4647,"num_object_clones":0,"num_object_copies":-9294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4647,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007898"},"pg_stats":[{"pgid":"2.f","version":"51'19289","reported_seq":23435,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.987848+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:29.987848+0000","last_peered":"2024-09-16T04:58:29.987848+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:29.987848+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1839104,"num_objects":449,"num_object_clones":0,"num_object_copies":898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1313,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19289,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2029,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-864}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19531","reported_seq":19657,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.872490+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:29.872490+0000","last_peered":"2024-09-16T04:58:29.872490+0000","last_clean":"2024-09-16T04:58:29.872490+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:29.872490+0000","last_undegraded":"2024-09-16T04:58:29.872490+0000","last_fullsized":"2024-09-16T04:58:29.872490+0000","mapping_epoch":35,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1896448,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16500,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19421","reported_seq":19571,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.844735+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:29.844735+0000","last_peered":"2024-09-16T04:58:29.844735+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:29.844735+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1785880,"num_objects":437,"num_object_clones":0,"num_object_copies":874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16160,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2882}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'19856","reported_seq":19937,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.820921+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:29.820921+0000","last_peered":"2024-09-16T04:58:29.820921+0000","last_clean":"2024-09-16T04:58:29.820921+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:29.820921+0000","last_undegraded":"2024-09-16T04:58:29.820921+0000","last_fullsized":"2024-09-16T04:58:29.820921+0000","mapping_epoch":35,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1998848,"num_objects":488,"num_object_clones":0,"num_object_copies":976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19856,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19684","reported_seq":19914,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.932838+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:29.932838+0000","last_peered":"2024-09-16T04:58:29.932838+0000","last_clean":"2024-09-16T04:58:29.932838+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:29.932838+0000","last_undegraded":"2024-09-16T04:58:29.932838+0000","last_fullsized":"2024-09-16T04:58:29.932838+0000","mapping_epoch":34,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1884160,"num_objects":460,"num_object_clones":0,"num_object_copies":920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16716,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19839","reported_seq":19983,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.923351+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:29.923351+0000","last_peered":"2024-09-16T04:58:29.923351+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:29.923351+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1941504,"num_objects":474,"num_object_clones":0,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2982}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19432","reported_seq":19591,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.895652+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:29.895652+0000","last_peered":"2024-09-16T04:58:29.895652+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:29.895652+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1753088,"num_objects":428,"num_object_clones":0,"num_object_copies":856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16087,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19573","reported_seq":19800,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.853488+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:29.853488+0000","last_peered":"2024-09-16T04:58:29.853488+0000","last_clean":"2024-09-16T04:58:29.853488+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:29.853488+0000","last_undegraded":"2024-09-16T04:58:29.853488+0000","last_fullsized":"2024-09-16T04:58:29.853488+0000","mapping_epoch":34,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1814528,"num_objects":443,"num_object_clones":0,"num_object_copies":886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16643,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19632","reported_seq":19871,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.737951+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:29.737951+0000","last_peered":"2024-09-16T04:58:29.737951+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:29.737951+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1839104,"num_objects":449,"num_object_clones":0,"num_object_copies":898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16383,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2872}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19771","reported_seq":19911,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.897975+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:29.897975+0000","last_peered":"2024-09-16T04:58:29.897975+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:29.897975+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1830912,"num_objects":447,"num_object_clones":0,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16426,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3018}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19534","reported_seq":26390,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.812312+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:29.812312+0000","last_peered":"2024-09-16T04:58:29.812312+0000","last_clean":"2024-09-16T04:58:29.812312+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:29.812312+0000","last_undegraded":"2024-09-16T04:58:29.812312+0000","last_fullsized":"2024-09-16T04:58:29.812312+0000","mapping_epoch":50,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:34.781909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823954,"num_objects":447,"num_object_clones":0,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19506,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19515","reported_seq":26382,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.861927+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:29.861927+0000","last_peered":"2024-09-16T04:58:29.861927+0000","last_clean":"2024-09-16T04:58:29.861927+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:29.861927+0000","last_undegraded":"2024-09-16T04:58:29.861927+0000","last_fullsized":"2024-09-16T04:58:29.861927+0000","mapping_epoch":46,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1896448,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16238,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19417","reported_seq":19526,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.822937+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:29.822937+0000","last_peered":"2024-09-16T04:58:29.822937+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:29.822937+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1880681,"num_objects":460,"num_object_clones":0,"num_object_copies":920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19416,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2947}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20070","reported_seq":20184,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.796787+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:29.796787+0000","last_peered":"2024-09-16T04:58:29.796787+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:29.796787+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1894203,"num_objects":465,"num_object_clones":0,"num_object_copies":930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20067,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3049}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'19735","reported_seq":26540,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.688193+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:29.688193+0000","last_peered":"2024-09-16T04:58:29.688193+0000","last_clean":"2024-09-16T04:58:29.688193+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:29.688193+0000","last_undegraded":"2024-09-16T04:58:29.688193+0000","last_fullsized":"2024-09-16T04:58:29.688193+0000","mapping_epoch":48,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1896448,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16489,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19954","reported_seq":20105,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.833379+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:29.833379+0000","last_peered":"2024-09-16T04:58:29.833379+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:29.833379+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1892969,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16723,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":29868279,"num_objects":7299,"num_object_clones":0,"num_object_copies":14598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279331,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12008,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":226594816,"data_stored":226520571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":100860,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":103280640,"data_stored":102986993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,6,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593835,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465924,"kb_used_data":23276,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93905916,"statfs":{"total":96636764160,"available":96159657984,"internally_reserved":0,"allocated":23834624,"data_stored":23485313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,0,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506484,"kb_used_data":100000,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93865356,"statfs":{"total":96636764160,"available":96118124544,"internally_reserved":0,"allocated":102400000,"data_stored":102110425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":101367808,"data_stored":101364329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22544384,"data_stored":22481169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102682624,"data_stored":102675073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:33.346 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:34.131 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:34.132 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:34.176 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-16T04:58:32.536686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29868298,"num_objects":7301,"num_object_clones":0,"num_object_copies":14602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279333,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12009,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160756,"ondisk_log_size":160756,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483972,"kb_used_data":224136,"kb_used_omap":12,"kb_used_meta":1259763,"kb_avail":281631548,"statfs":{"total":289910292480,"available":288390705152,"internally_reserved":0,"allocated":229515264,"data_stored":228582731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1289997313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,7,3,8,7,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19034112,"num_objects":-4647,"num_object_clones":0,"num_object_copies":-9294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4647,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007898"},"pg_stats":[{"pgid":"2.f","version":"51'19289","reported_seq":23435,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.987848+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:29.987848+0000","last_peered":"2024-09-16T04:58:29.987848+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:29.987848+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1839104,"num_objects":449,"num_object_clones":0,"num_object_copies":898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1313,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19289,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2029,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-864}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19531","reported_seq":19657,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.872490+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:29.872490+0000","last_peered":"2024-09-16T04:58:29.872490+0000","last_clean":"2024-09-16T04:58:29.872490+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:29.872490+0000","last_undegraded":"2024-09-16T04:58:29.872490+0000","last_fullsized":"2024-09-16T04:58:29.872490+0000","mapping_epoch":35,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1896448,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16500,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19421","reported_seq":19571,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.844735+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:29.844735+0000","last_peered":"2024-09-16T04:58:29.844735+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:29.844735+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1785880,"num_objects":437,"num_object_clones":0,"num_object_copies":874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16160,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2882}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'19856","reported_seq":19937,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.820921+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:29.820921+0000","last_peered":"2024-09-16T04:58:29.820921+0000","last_clean":"2024-09-16T04:58:29.820921+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:29.820921+0000","last_undegraded":"2024-09-16T04:58:29.820921+0000","last_fullsized":"2024-09-16T04:58:29.820921+0000","mapping_epoch":35,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1998848,"num_objects":488,"num_object_clones":0,"num_object_copies":976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19856,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19684","reported_seq":19914,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.932838+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:29.932838+0000","last_peered":"2024-09-16T04:58:29.932838+0000","last_clean":"2024-09-16T04:58:29.932838+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:29.932838+0000","last_undegraded":"2024-09-16T04:58:29.932838+0000","last_fullsized":"2024-09-16T04:58:29.932838+0000","mapping_epoch":34,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1884160,"num_objects":460,"num_object_clones":0,"num_object_copies":920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16716,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19839","reported_seq":19983,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.923351+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:29.923351+0000","last_peered":"2024-09-16T04:58:29.923351+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:29.923351+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1941504,"num_objects":474,"num_object_clones":0,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2982}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19432","reported_seq":19591,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.895652+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:29.895652+0000","last_peered":"2024-09-16T04:58:29.895652+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:29.895652+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1753088,"num_objects":428,"num_object_clones":0,"num_object_copies":856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16087,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19573","reported_seq":19800,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.853488+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:29.853488+0000","last_peered":"2024-09-16T04:58:29.853488+0000","last_clean":"2024-09-16T04:58:29.853488+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:29.853488+0000","last_undegraded":"2024-09-16T04:58:29.853488+0000","last_fullsized":"2024-09-16T04:58:29.853488+0000","mapping_epoch":34,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1814528,"num_objects":443,"num_object_clones":0,"num_object_copies":886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16643,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19632","reported_seq":19871,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.737951+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:29.737951+0000","last_peered":"2024-09-16T04:58:29.737951+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:29.737951+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1839104,"num_objects":449,"num_object_clones":0,"num_object_copies":898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16383,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2872}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19771","reported_seq":19911,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.897975+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:29.897975+0000","last_peered":"2024-09-16T04:58:29.897975+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:29.897975+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1830912,"num_objects":447,"num_object_clones":0,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16426,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3018}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19534","reported_seq":26390,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.812312+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:29.812312+0000","last_peered":"2024-09-16T04:58:29.812312+0000","last_clean":"2024-09-16T04:58:29.812312+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:29.812312+0000","last_undegraded":"2024-09-16T04:58:29.812312+0000","last_fullsized":"2024-09-16T04:58:29.812312+0000","mapping_epoch":50,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:34.781909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823954,"num_objects":447,"num_object_clones":0,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19506,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19515","reported_seq":26382,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.861927+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:29.861927+0000","last_peered":"2024-09-16T04:58:29.861927+0000","last_clean":"2024-09-16T04:58:29.861927+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:29.861927+0000","last_undegraded":"2024-09-16T04:58:29.861927+0000","last_fullsized":"2024-09-16T04:58:29.861927+0000","mapping_epoch":46,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1896448,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16238,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19417","reported_seq":19526,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.822937+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:29.822937+0000","last_peered":"2024-09-16T04:58:29.822937+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:29.822937+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1880681,"num_objects":460,"num_object_clones":0,"num_object_copies":920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19416,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2947}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20070","reported_seq":20184,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.796787+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:29.796787+0000","last_peered":"2024-09-16T04:58:29.796787+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:29.796787+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1894203,"num_objects":465,"num_object_clones":0,"num_object_copies":930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20067,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3049}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'19735","reported_seq":26540,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.688193+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:29.688193+0000","last_peered":"2024-09-16T04:58:29.688193+0000","last_clean":"2024-09-16T04:58:29.688193+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:29.688193+0000","last_undegraded":"2024-09-16T04:58:29.688193+0000","last_fullsized":"2024-09-16T04:58:29.688193+0000","mapping_epoch":48,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1896448,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16489,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19954","reported_seq":20105,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.833379+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:29.833379+0000","last_peered":"2024-09-16T04:58:29.833379+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:29.833379+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1892969,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16723,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":29868279,"num_objects":7299,"num_object_clones":0,"num_object_copies":14598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279331,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12008,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":226594816,"data_stored":226520571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":100860,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":103280640,"data_stored":102986993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,6,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593835,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465924,"kb_used_data":23276,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93905916,"statfs":{"total":96636764160,"available":96159657984,"internally_reserved":0,"allocated":23834624,"data_stored":23485313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,0,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506484,"kb_used_data":100000,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93865356,"statfs":{"total":96636764160,"available":96118124544,"internally_reserved":0,"allocated":102400000,"data_stored":102110425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":101367808,"data_stored":101364329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22544384,"data_stored":22481169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102682624,"data_stored":102675073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:34.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:58:34.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'19289", 'reported_seq': 23435, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:29.987848+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:58:29.987848+0000', 'last_peered': '2024-09-16T04:58:29.987848+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:58:29.987848+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'9200", 'ondisk_log_start': "47'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1839104, 'num_objects': 449, 'num_object_clones': 0, 'num_object_copies': 898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1313, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 449, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19289, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2029, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -864}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:34.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:58:34.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19421", 'reported_seq': 19571, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:29.844735+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:29.844735+0000', 'last_peered': '2024-09-16T04:58:29.844735+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:29.844735+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9400", 'ondisk_log_start': "47'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1785880, 'num_objects': 437, 'num_object_clones': 0, 'num_object_copies': 874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 437, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16160, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2882}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:34.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:58:34.178 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'19839", 'reported_seq': 19983, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:29.923351+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:29.923351+0000', 'last_peered': '2024-09-16T04:58:29.923351+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:29.923351+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1941504, 'num_objects': 474, 'num_object_clones': 0, 'num_object_copies': 948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2982}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:34.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:58:34.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19432", 'reported_seq': 19591, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:29.895652+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:29.895652+0000', 'last_peered': '2024-09-16T04:58:29.895652+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:29.895652+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9400", 'ondisk_log_start': "47'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1753088, 'num_objects': 428, 'num_object_clones': 0, 'num_object_copies': 856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16087, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2971}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:34.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:58:34.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'19632", 'reported_seq': 19871, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:29.737951+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:29.737951+0000', 'last_peered': '2024-09-16T04:58:29.737951+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:29.737951+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'9600", 'ondisk_log_start': "47'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1839104, 'num_objects': 449, 'num_object_clones': 0, 'num_object_copies': 898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 449, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16383, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2872}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:34.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:58:34.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'19771", 'reported_seq': 19911, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:29.897975+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:29.897975+0000', 'last_peered': '2024-09-16T04:58:29.897975+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:29.897975+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'9700", 'ondisk_log_start': "47'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1830912, 'num_objects': 447, 'num_object_clones': 0, 'num_object_copies': 894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 447, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16426, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3018}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:34.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:58:34.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19417", 'reported_seq': 19526, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:29.822937+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:29.822937+0000', 'last_peered': '2024-09-16T04:58:29.822937+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:29.822937+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9400", 'ondisk_log_start': "47'9400", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1880681, 'num_objects': 460, 'num_object_clones': 0, 'num_object_copies': 920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 460, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19416, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2947}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:34.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:58:34.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'20070", 'reported_seq': 20184, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:29.796787+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:58:29.796787+0000', 'last_peered': '2024-09-16T04:58:29.796787+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:58:29.796787+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1894203, 'num_objects': 465, 'num_object_clones': 0, 'num_object_copies': 930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 465, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20067, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3049}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:34.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:58:34.180 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'19954", 'reported_seq': 20105, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:29.833379+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:29.833379+0000', 'last_peered': '2024-09-16T04:58:29.833379+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:29.833379+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'9900", 'ondisk_log_start': "47'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1892969, 'num_objects': 463, 'num_object_clones': 0, 'num_object_copies': 926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 463, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16723, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3061}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:34.180 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:58:35.173 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:35.173 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28734/14602 objects degraded (196.781%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":662,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":7301,"data_bytes":29868298,"bytes_used":1519587328,"bytes_avail":288390705152,"bytes_total":289910292480,"degraded_objects":28734,"degraded_total":14602,"degraded_ratio":1.9678126284070678,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":387},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:58:35.257 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:58:35.258 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:36.040 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:36.087 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-16T04:58:34.537330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29868298,"num_objects":7301,"num_object_clones":0,"num_object_copies":14602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279333,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12009,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160756,"ondisk_log_size":160756,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483972,"kb_used_data":224136,"kb_used_omap":12,"kb_used_meta":1259763,"kb_avail":281631548,"statfs":{"total":289910292480,"available":288390705152,"internally_reserved":0,"allocated":229515264,"data_stored":228582731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1289997313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,7,3,8,7,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19034112,"num_objects":-4647,"num_object_clones":0,"num_object_copies":-9294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4647,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004542"},"pg_stats":[{"pgid":"2.f","version":"51'19289","reported_seq":23435,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.987848+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:29.987848+0000","last_peered":"2024-09-16T04:58:29.987848+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:29.987848+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9200","ondisk_log_start":"47'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1839104,"num_objects":449,"num_object_clones":0,"num_object_copies":898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1313,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19289,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2029,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-864}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19531","reported_seq":19657,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.872490+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:29.872490+0000","last_peered":"2024-09-16T04:58:29.872490+0000","last_clean":"2024-09-16T04:58:29.872490+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:29.872490+0000","last_undegraded":"2024-09-16T04:58:29.872490+0000","last_fullsized":"2024-09-16T04:58:29.872490+0000","mapping_epoch":35,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1896448,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16500,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19421","reported_seq":19571,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.844735+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:29.844735+0000","last_peered":"2024-09-16T04:58:29.844735+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:29.844735+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1785880,"num_objects":437,"num_object_clones":0,"num_object_copies":874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16160,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2882}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'19856","reported_seq":19937,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.820921+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:29.820921+0000","last_peered":"2024-09-16T04:58:29.820921+0000","last_clean":"2024-09-16T04:58:29.820921+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:29.820921+0000","last_undegraded":"2024-09-16T04:58:29.820921+0000","last_fullsized":"2024-09-16T04:58:29.820921+0000","mapping_epoch":35,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1998848,"num_objects":488,"num_object_clones":0,"num_object_copies":976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19856,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19684","reported_seq":19914,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.932838+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:29.932838+0000","last_peered":"2024-09-16T04:58:29.932838+0000","last_clean":"2024-09-16T04:58:29.932838+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:29.932838+0000","last_undegraded":"2024-09-16T04:58:29.932838+0000","last_fullsized":"2024-09-16T04:58:29.932838+0000","mapping_epoch":34,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1884160,"num_objects":460,"num_object_clones":0,"num_object_copies":920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16716,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19839","reported_seq":19983,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.923351+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:29.923351+0000","last_peered":"2024-09-16T04:58:29.923351+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:29.923351+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1941504,"num_objects":474,"num_object_clones":0,"num_object_copies":948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2982}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19432","reported_seq":19591,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.895652+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:29.895652+0000","last_peered":"2024-09-16T04:58:29.895652+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:29.895652+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1753088,"num_objects":428,"num_object_clones":0,"num_object_copies":856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16087,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19573","reported_seq":19800,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.853488+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:29.853488+0000","last_peered":"2024-09-16T04:58:29.853488+0000","last_clean":"2024-09-16T04:58:29.853488+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:29.853488+0000","last_undegraded":"2024-09-16T04:58:29.853488+0000","last_fullsized":"2024-09-16T04:58:29.853488+0000","mapping_epoch":34,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1814528,"num_objects":443,"num_object_clones":0,"num_object_copies":886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16643,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19632","reported_seq":19871,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.737951+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:29.737951+0000","last_peered":"2024-09-16T04:58:29.737951+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:29.737951+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1839104,"num_objects":449,"num_object_clones":0,"num_object_copies":898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16383,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2872}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19771","reported_seq":19911,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.897975+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:29.897975+0000","last_peered":"2024-09-16T04:58:29.897975+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:29.897975+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1830912,"num_objects":447,"num_object_clones":0,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16426,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3018}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19534","reported_seq":26390,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.812312+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:29.812312+0000","last_peered":"2024-09-16T04:58:29.812312+0000","last_clean":"2024-09-16T04:58:29.812312+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:29.812312+0000","last_undegraded":"2024-09-16T04:58:29.812312+0000","last_fullsized":"2024-09-16T04:58:29.812312+0000","mapping_epoch":50,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:34.781909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1823954,"num_objects":447,"num_object_clones":0,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19506,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19515","reported_seq":26382,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.861927+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:29.861927+0000","last_peered":"2024-09-16T04:58:29.861927+0000","last_clean":"2024-09-16T04:58:29.861927+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:29.861927+0000","last_undegraded":"2024-09-16T04:58:29.861927+0000","last_fullsized":"2024-09-16T04:58:29.861927+0000","mapping_epoch":46,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1896448,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16238,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19417","reported_seq":19526,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.822937+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:29.822937+0000","last_peered":"2024-09-16T04:58:29.822937+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:29.822937+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1880681,"num_objects":460,"num_object_clones":0,"num_object_copies":920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19416,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2947}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20070","reported_seq":20184,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.796787+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:29.796787+0000","last_peered":"2024-09-16T04:58:29.796787+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:29.796787+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1894203,"num_objects":465,"num_object_clones":0,"num_object_copies":930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20067,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3049}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'19735","reported_seq":26540,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:29.688193+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:29.688193+0000","last_peered":"2024-09-16T04:58:29.688193+0000","last_clean":"2024-09-16T04:58:29.688193+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:29.688193+0000","last_undegraded":"2024-09-16T04:58:29.688193+0000","last_fullsized":"2024-09-16T04:58:29.688193+0000","mapping_epoch":48,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1896448,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16489,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'19954","reported_seq":20105,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:29.833379+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:29.833379+0000","last_peered":"2024-09-16T04:58:29.833379+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:29.833379+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1892969,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16723,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":29868279,"num_objects":7299,"num_object_clones":0,"num_object_copies":14598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279331,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12008,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":226594816,"data_stored":226520571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511564,"kb_used_data":100860,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93860276,"statfs":{"total":96636764160,"available":96112922624,"internally_reserved":0,"allocated":103280640,"data_stored":102986993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,6,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593835,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465924,"kb_used_data":23276,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93905916,"statfs":{"total":96636764160,"available":96159657984,"internally_reserved":0,"allocated":23834624,"data_stored":23485313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":453242542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,0,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506484,"kb_used_data":100000,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93865356,"statfs":{"total":96636764160,"available":96118124544,"internally_reserved":0,"allocated":102400000,"data_stored":102110425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":101367808,"data_stored":101364329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22544384,"data_stored":22481169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102682624,"data_stored":102675073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:39.089 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:40.069 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:40.143 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-16T04:58:38.538229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20111626,"num_objects":4919,"num_object_clones":0,"num_object_copies":9838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4919,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281715,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12092,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160938,"ondisk_log_size":160938,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477576,"kb_used_data":205272,"kb_used_omap":12,"kb_used_meta":1272243,"kb_avail":281637944,"statfs":{"total":289910292480,"available":288397254656,"internally_reserved":0,"allocated":210198528,"data_stored":209265995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1302776833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,5,7,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19316736,"num_objects":-4716,"num_object_clones":0,"num_object_copies":-9432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4716,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004577"},"pg_stats":[{"pgid":"2.f","version":"51'19465","reported_seq":23778,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:34.950282+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:34.950282+0000","last_peered":"2024-09-16T04:58:34.950282+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:34.950282+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9400","ondisk_log_start":"47'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1118208,"num_objects":273,"num_object_clones":0,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19465,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2112,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-957}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19683","reported_seq":19810,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:34.876343+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:34.876343+0000","last_peered":"2024-09-16T04:58:34.876343+0000","last_clean":"2024-09-16T04:58:34.876343+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:34.876343+0000","last_undegraded":"2024-09-16T04:58:34.876343+0000","last_fullsized":"2024-09-16T04:58:34.876343+0000","mapping_epoch":35,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1273856,"num_objects":311,"num_object_clones":0,"num_object_copies":622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16652,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19570","reported_seq":19719,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:34.893668+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:34.893668+0000","last_peered":"2024-09-16T04:58:34.893668+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:34.893668+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175576,"num_objects":288,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16309,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3031}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20027","reported_seq":20108,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:34.937813+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:34.937813+0000","last_peered":"2024-09-16T04:58:34.937813+0000","last_clean":"2024-09-16T04:58:34.937813+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:34.937813+0000","last_undegraded":"2024-09-16T04:58:34.937813+0000","last_fullsized":"2024-09-16T04:58:34.937813+0000","mapping_epoch":35,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1298432,"num_objects":317,"num_object_clones":0,"num_object_copies":634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20027,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19835","reported_seq":20064,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:34.992977+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:34.992977+0000","last_peered":"2024-09-16T04:58:34.992977+0000","last_clean":"2024-09-16T04:58:34.992977+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:34.992977+0000","last_undegraded":"2024-09-16T04:58:34.992977+0000","last_fullsized":"2024-09-16T04:58:34.992977+0000","mapping_epoch":34,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1265664,"num_objects":309,"num_object_clones":0,"num_object_copies":618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16867,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'19997","reported_seq":20141,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:34.779371+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:34.779371+0000","last_peered":"2024-09-16T04:58:34.779371+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:34.779371+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294336,"num_objects":316,"num_object_clones":0,"num_object_copies":632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16990,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3140}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19567","reported_seq":19726,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:35.003984+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:35.003984+0000","last_peered":"2024-09-16T04:58:35.003984+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:35.003984+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1200128,"num_objects":293,"num_object_clones":0,"num_object_copies":586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16222,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3106}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19711","reported_seq":19938,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:34.965925+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:34.965925+0000","last_peered":"2024-09-16T04:58:34.965925+0000","last_clean":"2024-09-16T04:58:34.965925+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:34.965925+0000","last_undegraded":"2024-09-16T04:58:34.965925+0000","last_fullsized":"2024-09-16T04:58:34.965925+0000","mapping_epoch":34,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1249280,"num_objects":305,"num_object_clones":0,"num_object_copies":610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16781,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19779","reported_seq":20018,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:34.872785+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:34.872785+0000","last_peered":"2024-09-16T04:58:34.872785+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:34.872785+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1236992,"num_objects":302,"num_object_clones":0,"num_object_copies":604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16530,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3019}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'19899","reported_seq":20040,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:34.886973+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:34.886973+0000","last_peered":"2024-09-16T04:58:34.886973+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:34.886973+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1306624,"num_objects":319,"num_object_clones":0,"num_object_copies":638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16554,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19692","reported_seq":26548,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:34.934183+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:34.934183+0000","last_peered":"2024-09-16T04:58:34.934183+0000","last_clean":"2024-09-16T04:58:34.934183+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:34.934183+0000","last_undegraded":"2024-09-16T04:58:34.934183+0000","last_fullsized":"2024-09-16T04:58:34.934183+0000","mapping_epoch":50,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1176786,"num_objects":289,"num_object_clones":0,"num_object_copies":578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19664,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19652","reported_seq":26519,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:34.873325+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:34.873325+0000","last_peered":"2024-09-16T04:58:34.873325+0000","last_clean":"2024-09-16T04:58:34.873325+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:34.873325+0000","last_undegraded":"2024-09-16T04:58:34.873325+0000","last_fullsized":"2024-09-16T04:58:34.873325+0000","mapping_epoch":46,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1335296,"num_objects":326,"num_object_clones":0,"num_object_copies":652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16375,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19550","reported_seq":19659,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:34.888591+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:34.888591+0000","last_peered":"2024-09-16T04:58:34.888591+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:34.888591+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1335913,"num_objects":327,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19549,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3080}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20229","reported_seq":20343,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:34.936671+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:34.936671+0000","last_peered":"2024-09-16T04:58:34.936671+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:34.936671+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1242939,"num_objects":306,"num_object_clones":0,"num_object_copies":612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20226,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3208}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'19868","reported_seq":26673,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:34.656473+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:34.656473+0000","last_peered":"2024-09-16T04:58:34.656473+0000","last_clean":"2024-09-16T04:58:34.656473+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:34.656473+0000","last_undegraded":"2024-09-16T04:58:34.656473+0000","last_fullsized":"2024-09-16T04:58:34.656473+0000","mapping_epoch":48,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1351680,"num_objects":330,"num_object_clones":0,"num_object_copies":660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16622,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20111","reported_seq":20262,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:34.985296+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:34.985296+0000","last_peered":"2024-09-16T04:58:34.985296+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:34.985296+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1249897,"num_objects":306,"num_object_clones":0,"num_object_copies":612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16880,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3218}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":20111607,"num_objects":4917,"num_object_clones":0,"num_object_copies":9834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281713,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12091,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":207138816,"data_stored":207064571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160935,"ondisk_log_size":160935,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509880,"kb_used_data":95016,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93861960,"statfs":{"total":96636764160,"available":96114647040,"internally_reserved":0,"allocated":97296384,"data_stored":97002737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":424799913},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593836,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463136,"kb_used_data":16336,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93908704,"statfs":{"total":96636764160,"available":96162512896,"internally_reserved":0,"allocated":16728064,"data_stored":16378753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":457502382},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,2,5,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":504560,"kb_used_data":93920,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93867280,"statfs":{"total":96636764160,"available":96120094720,"internally_reserved":0,"allocated":96174080,"data_stored":95884505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":95281152,"data_stored":95277673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15257600,"data_stored":15194385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":96600064,"data_stored":96592513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:40.144 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:41.095 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:41.095 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:41.145 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-09-16T04:58:40.540338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10309898,"num_objects":2526,"num_object_clones":0,"num_object_copies":5052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284108,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12200,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501628,"kb_used_data":186304,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281613892,"statfs":{"total":289910292480,"available":288372625408,"internally_reserved":0,"allocated":190775296,"data_stored":189842763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1346817025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,2,5,1,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29118464,"num_objects":-7109,"num_object_clones":0,"num_object_copies":-14218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7109,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006211"},"pg_stats":[{"pgid":"2.f","version":"51'19590","reported_seq":24119,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:40.039693+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:40.039693+0000","last_peered":"2024-09-16T04:58:40.039693+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:40.039693+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":606208,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19590,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2220,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-974}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19833","reported_seq":19960,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.956761+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:39.956761+0000","last_peered":"2024-09-16T04:58:39.956761+0000","last_clean":"2024-09-16T04:58:39.956761+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:39.956761+0000","last_undegraded":"2024-09-16T04:58:39.956761+0000","last_fullsized":"2024-09-16T04:58:39.956761+0000","mapping_epoch":35,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":659456,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16802,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19695","reported_seq":19845,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.893379+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:39.893379+0000","last_peered":"2024-09-16T04:58:39.893379+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:39.893379+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":663576,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16434,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3156}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20182","reported_seq":20263,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.948065+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:39.948065+0000","last_peered":"2024-09-16T04:58:39.948065+0000","last_clean":"2024-09-16T04:58:39.948065+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:39.948065+0000","last_undegraded":"2024-09-16T04:58:39.948065+0000","last_fullsized":"2024-09-16T04:58:39.948065+0000","mapping_epoch":35,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":663552,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20182,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19999","reported_seq":20229,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.974676+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:39.974676+0000","last_peered":"2024-09-16T04:58:39.974676+0000","last_clean":"2024-09-16T04:58:39.974676+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:39.974676+0000","last_undegraded":"2024-09-16T04:58:39.974676+0000","last_fullsized":"2024-09-16T04:58:39.974676+0000","mapping_epoch":34,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":593920,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17031,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20153","reported_seq":20297,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.993590+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:39.993590+0000","last_peered":"2024-09-16T04:58:39.993590+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:39.993590+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17146,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3296}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19715","reported_seq":19874,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.968866+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:39.968866+0000","last_peered":"2024-09-16T04:58:39.968866+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:39.968866+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":593920,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16370,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3254}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19869","reported_seq":20096,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.949810+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:39.949810+0000","last_peered":"2024-09-16T04:58:39.949810+0000","last_clean":"2024-09-16T04:58:39.949810+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:39.949810+0000","last_undegraded":"2024-09-16T04:58:39.949810+0000","last_fullsized":"2024-09-16T04:58:39.949810+0000","mapping_epoch":34,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602112,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16939,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19930","reported_seq":20169,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.950022+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:39.950022+0000","last_peered":"2024-09-16T04:58:39.950022+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:39.950022+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":618496,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16681,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3170}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20066","reported_seq":20207,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:40.007827+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:40.007827+0000","last_peered":"2024-09-16T04:58:40.007827+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:40.007827+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622592,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16721,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3313}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19829","reported_seq":26685,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.794718+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:39.794718+0000","last_peered":"2024-09-16T04:58:39.794718+0000","last_clean":"2024-09-16T04:58:39.794718+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:39.794718+0000","last_undegraded":"2024-09-16T04:58:39.794718+0000","last_fullsized":"2024-09-16T04:58:39.794718+0000","mapping_epoch":50,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:40.704828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":615634,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19801,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19797","reported_seq":26664,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.936089+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:39.936089+0000","last_peered":"2024-09-16T04:58:39.936089+0000","last_clean":"2024-09-16T04:58:39.936089+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:39.936089+0000","last_undegraded":"2024-09-16T04:58:39.936089+0000","last_fullsized":"2024-09-16T04:58:39.936089+0000","mapping_epoch":46,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19717","reported_seq":19825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.893450+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:39.893450+0000","last_peered":"2024-09-16T04:58:39.893450+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:39.893450+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":651881,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19716,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3247}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20371","reported_seq":20485,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.922010+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:39.922010+0000","last_peered":"2024-09-16T04:58:39.922010+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:39.922010+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661307,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20368,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3350}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20023","reported_seq":26828,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.827091+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:39.827091+0000","last_peered":"2024-09-16T04:58:39.827091+0000","last_clean":"2024-09-16T04:58:39.827091+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:39.827091+0000","last_undegraded":"2024-09-16T04:58:39.827091+0000","last_fullsized":"2024-09-16T04:58:39.827091+0000","mapping_epoch":48,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":716800,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16777,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20259","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.994482+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:39.994482+0000","last_peered":"2024-09-16T04:58:39.994482+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:39.994482+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643689,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17028,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":10309879,"num_objects":2524,"num_object_clones":0,"num_object_copies":5048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284106,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12199,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":187559936,"data_stored":187485691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512184,"kb_used_data":88996,"kb_used_omap":4,"kb_used_meta":423163,"kb_avail":93859656,"statfs":{"total":96636764160,"available":96112287744,"internally_reserved":0,"allocated":91131904,"data_stored":90838257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":433319593},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,1,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593837,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482520,"kb_used_data":9348,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93889320,"statfs":{"total":96636764160,"available":96142663680,"internally_reserved":0,"allocated":9572352,"data_stored":9223041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506924,"kb_used_data":87960,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93864916,"statfs":{"total":96636764160,"available":96117673984,"internally_reserved":0,"allocated":90071040,"data_stored":89781465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":89055232,"data_stored":89051753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8048640,"data_stored":7985425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90456064,"data_stored":90448513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:41.146 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:58:41.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'19590", 'reported_seq': 24119, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:40.039693+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:58:40.039693+0000', 'last_peered': '2024-09-16T04:58:40.039693+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:58:40.039693+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'9500", 'ondisk_log_start': "47'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 606208, 'num_objects': 148, 'num_object_clones': 0, 'num_object_copies': 296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1122, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 148, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19590, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2220, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -974}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:41.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:58:41.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19695", 'reported_seq': 19845, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:39.893379+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:39.893379+0000', 'last_peered': '2024-09-16T04:58:39.893379+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:39.893379+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9600", 'ondisk_log_start': "47'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 663576, 'num_objects': 163, 'num_object_clones': 0, 'num_object_copies': 326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 163, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16434, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3156}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:41.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:58:41.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20153", 'reported_seq': 20297, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:39.993590+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:39.993590+0000', 'last_peered': '2024-09-16T04:58:39.993590+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:39.993590+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10100", 'ondisk_log_start': "47'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 655360, 'num_objects': 160, 'num_object_clones': 0, 'num_object_copies': 320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17146, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3296}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:41.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:58:41.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19715", 'reported_seq': 19874, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:39.968866+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:39.968866+0000', 'last_peered': '2024-09-16T04:58:39.968866+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:39.968866+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9700", 'ondisk_log_start': "47'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 593920, 'num_objects': 145, 'num_object_clones': 0, 'num_object_copies': 290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 145, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16370, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3254}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:41.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:58:41.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'19930", 'reported_seq': 20169, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:39.950022+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:39.950022+0000', 'last_peered': '2024-09-16T04:58:39.950022+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:39.950022+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'9900", 'ondisk_log_start': "47'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 618496, 'num_objects': 151, 'num_object_clones': 0, 'num_object_copies': 302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 151, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16681, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3170}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:41.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:58:41.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20066", 'reported_seq': 20207, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:40.007827+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:40.007827+0000', 'last_peered': '2024-09-16T04:58:40.007827+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:40.007827+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 622592, 'num_objects': 152, 'num_object_clones': 0, 'num_object_copies': 304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 152, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16721, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3313}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:41.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:58:41.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19717", 'reported_seq': 19825, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:39.893450+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:39.893450+0000', 'last_peered': '2024-09-16T04:58:39.893450+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:39.893450+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9700", 'ondisk_log_start': "47'9700", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 651881, 'num_objects': 160, 'num_object_clones': 0, 'num_object_copies': 320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19716, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3247}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:41.148 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:58:41.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'20371", 'reported_seq': 20485, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:39.922010+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:58:39.922010+0000', 'last_peered': '2024-09-16T04:58:39.922010+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:58:39.922010+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 661307, 'num_objects': 164, 'num_object_clones': 0, 'num_object_copies': 328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 164, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20368, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3350}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:41.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:58:41.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20259", 'reported_seq': 20410, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:39.994482+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:39.994482+0000', 'last_peered': '2024-09-16T04:58:39.994482+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:39.994482+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 643689, 'num_objects': 158, 'num_object_clones': 0, 'num_object_copies': 316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17028, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3366}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:41.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:58:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:42.354 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28543/5052 objects degraded (564.984%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":670,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":2526,"data_bytes":10309898,"bytes_used":1537667072,"bytes_avail":288372625408,"bytes_total":289910292480,"degraded_objects":28543,"degraded_total":5052,"degraded_ratio":5.6498416468725257,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":592},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:58:42.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:58:42.395 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:43.213 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:43.215 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:43.254 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-16T04:58:42.540793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10309898,"num_objects":2526,"num_object_clones":0,"num_object_copies":5052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284108,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12200,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160931,"ondisk_log_size":160931,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501628,"kb_used_data":186304,"kb_used_omap":12,"kb_used_meta":1315251,"kb_avail":281613892,"statfs":{"total":289910292480,"available":288372625408,"internally_reserved":0,"allocated":190775296,"data_stored":189842763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1346817025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,2,5,1,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19558400,"num_objects":-4775,"num_object_clones":0,"num_object_copies":-9550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4775,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004615"},"pg_stats":[{"pgid":"2.f","version":"51'19590","reported_seq":24119,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:40.039693+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:40.039693+0000","last_peered":"2024-09-16T04:58:40.039693+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:40.039693+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9500","ondisk_log_start":"47'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":606208,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19590,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2220,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-974}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19833","reported_seq":19960,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.956761+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:39.956761+0000","last_peered":"2024-09-16T04:58:39.956761+0000","last_clean":"2024-09-16T04:58:39.956761+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:39.956761+0000","last_undegraded":"2024-09-16T04:58:39.956761+0000","last_fullsized":"2024-09-16T04:58:39.956761+0000","mapping_epoch":35,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":659456,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16802,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19695","reported_seq":19845,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.893379+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:39.893379+0000","last_peered":"2024-09-16T04:58:39.893379+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:39.893379+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9600","ondisk_log_start":"47'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":663576,"num_objects":163,"num_object_clones":0,"num_object_copies":326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16434,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3156}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20182","reported_seq":20263,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.948065+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:39.948065+0000","last_peered":"2024-09-16T04:58:39.948065+0000","last_clean":"2024-09-16T04:58:39.948065+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:39.948065+0000","last_undegraded":"2024-09-16T04:58:39.948065+0000","last_fullsized":"2024-09-16T04:58:39.948065+0000","mapping_epoch":35,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":663552,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20182,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'19999","reported_seq":20229,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.974676+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:39.974676+0000","last_peered":"2024-09-16T04:58:39.974676+0000","last_clean":"2024-09-16T04:58:39.974676+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:39.974676+0000","last_undegraded":"2024-09-16T04:58:39.974676+0000","last_fullsized":"2024-09-16T04:58:39.974676+0000","mapping_epoch":34,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":593920,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17031,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20153","reported_seq":20297,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.993590+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:39.993590+0000","last_peered":"2024-09-16T04:58:39.993590+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:39.993590+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17146,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3296}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19715","reported_seq":19874,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.968866+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:39.968866+0000","last_peered":"2024-09-16T04:58:39.968866+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:39.968866+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":593920,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16370,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3254}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'19869","reported_seq":20096,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.949810+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:39.949810+0000","last_peered":"2024-09-16T04:58:39.949810+0000","last_clean":"2024-09-16T04:58:39.949810+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:39.949810+0000","last_undegraded":"2024-09-16T04:58:39.949810+0000","last_fullsized":"2024-09-16T04:58:39.949810+0000","mapping_epoch":34,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602112,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16939,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'19930","reported_seq":20169,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.950022+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:39.950022+0000","last_peered":"2024-09-16T04:58:39.950022+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:39.950022+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":618496,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16681,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3170}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20066","reported_seq":20207,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:40.007827+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:40.007827+0000","last_peered":"2024-09-16T04:58:40.007827+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:40.007827+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622592,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16721,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3313}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19829","reported_seq":26685,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.794718+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:39.794718+0000","last_peered":"2024-09-16T04:58:39.794718+0000","last_clean":"2024-09-16T04:58:39.794718+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:39.794718+0000","last_undegraded":"2024-09-16T04:58:39.794718+0000","last_fullsized":"2024-09-16T04:58:39.794718+0000","mapping_epoch":50,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:40.704828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":615634,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19801,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19797","reported_seq":26664,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.936089+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:39.936089+0000","last_peered":"2024-09-16T04:58:39.936089+0000","last_clean":"2024-09-16T04:58:39.936089+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:39.936089+0000","last_undegraded":"2024-09-16T04:58:39.936089+0000","last_fullsized":"2024-09-16T04:58:39.936089+0000","mapping_epoch":46,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":741376,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16520,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19717","reported_seq":19825,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.893450+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:39.893450+0000","last_peered":"2024-09-16T04:58:39.893450+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:39.893450+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":651881,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19716,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3247}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20371","reported_seq":20485,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.922010+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:39.922010+0000","last_peered":"2024-09-16T04:58:39.922010+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:39.922010+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661307,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20368,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3350}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20023","reported_seq":26828,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:39.827091+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:39.827091+0000","last_peered":"2024-09-16T04:58:39.827091+0000","last_clean":"2024-09-16T04:58:39.827091+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:39.827091+0000","last_undegraded":"2024-09-16T04:58:39.827091+0000","last_fullsized":"2024-09-16T04:58:39.827091+0000","mapping_epoch":48,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":716800,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16777,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20259","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:39.994482+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:39.994482+0000","last_peered":"2024-09-16T04:58:39.994482+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:39.994482+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643689,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17028,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":10309879,"num_objects":2524,"num_object_clones":0,"num_object_copies":5048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2524,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":284106,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12199,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":187559936,"data_stored":187485691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512184,"kb_used_data":88996,"kb_used_omap":4,"kb_used_meta":423163,"kb_avail":93859656,"statfs":{"total":96636764160,"available":96112287744,"internally_reserved":0,"allocated":91131904,"data_stored":90838257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":433319593},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,1,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593837,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482520,"kb_used_data":9348,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93889320,"statfs":{"total":96636764160,"available":96142663680,"internally_reserved":0,"allocated":9572352,"data_stored":9223041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506924,"kb_used_data":87960,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93864916,"statfs":{"total":96636764160,"available":96117673984,"internally_reserved":0,"allocated":90071040,"data_stored":89781465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":428994218},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":89055232,"data_stored":89051753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8048640,"data_stored":7985425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90456064,"data_stored":90448513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:45.286 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Removed 160690 objects 2024-09-16T04:58:45.287 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi012.stdout:Clean up completed and total clean up time :348.125 2024-09-16T04:58:46.256 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:46.616 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:46.617 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:46.633 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-16T04:58:46.541612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":532746,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286495,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12311,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1449760,"kb_used_data":167708,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281665760,"statfs":{"total":289910292480,"available":288425738240,"internally_reserved":0,"allocated":171732992,"data_stored":170800459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,8,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29335552,"num_objects":-7162,"num_object_clones":0,"num_object_copies":-14324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7162,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004282"},"pg_stats":[{"pgid":"2.f","version":"51'19734","reported_seq":24485,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.086093+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:45.086093+0000","last_peered":"2024-09-16T04:58:45.086093+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:45.086093+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16384,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1011,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19734,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2331,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1007}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19985","reported_seq":20112,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.900860+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:44.900860+0000","last_peered":"2024-09-16T04:58:44.900860+0000","last_clean":"2024-09-16T04:58:44.900860+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:44.900860+0000","last_undegraded":"2024-09-16T04:58:44.900860+0000","last_fullsized":"2024-09-16T04:58:44.900860+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36864,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16954,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19851","reported_seq":20001,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:44.859402+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:44.859402+0000","last_peered":"2024-09-16T04:58:44.859402+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:44.859402+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24600,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16590,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20331","reported_seq":20411,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.989593+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:44.989593+0000","last_peered":"2024-09-16T04:58:44.989593+0000","last_clean":"2024-09-16T04:58:44.989593+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:44.989593+0000","last_undegraded":"2024-09-16T04:58:44.989593+0000","last_fullsized":"2024-09-16T04:58:44.989593+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53248,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20331,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20142","reported_seq":20372,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.071041+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:45.071041+0000","last_peered":"2024-09-16T04:58:45.071041+0000","last_clean":"2024-09-16T04:58:45.071041+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:45.071041+0000","last_undegraded":"2024-09-16T04:58:45.071041+0000","last_fullsized":"2024-09-16T04:58:45.071041+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8192,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17174,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20304","reported_seq":20448,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.011657+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:45.011657+0000","last_peered":"2024-09-16T04:58:45.011657+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:45.011657+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36864,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17297,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3447}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19851","reported_seq":20010,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.022432+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:45.022432+0000","last_peered":"2024-09-16T04:58:45.022432+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:45.022432+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36864,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16506,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3390}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20012","reported_seq":20239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.986469+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:44.986469+0000","last_peered":"2024-09-16T04:58:44.986469+0000","last_clean":"2024-09-16T04:58:44.986469+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:44.986469+0000","last_undegraded":"2024-09-16T04:58:44.986469+0000","last_fullsized":"2024-09-16T04:58:44.986469+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16384,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17082,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20073","reported_seq":20312,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:44.913407+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:44.913407+0000","last_peered":"2024-09-16T04:58:44.913407+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:44.913407+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32768,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16824,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3313}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20206","reported_seq":20346,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.039215+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:45.039215+0000","last_peered":"2024-09-16T04:58:45.039215+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:45.039215+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49152,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16861,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3453}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19973","reported_seq":26829,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.974985+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:44.974985+0000","last_peered":"2024-09-16T04:58:44.974985+0000","last_clean":"2024-09-16T04:58:44.974985+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:44.974985+0000","last_undegraded":"2024-09-16T04:58:44.974985+0000","last_fullsized":"2024-09-16T04:58:44.974985+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:45.744684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25810,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19945,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19972","reported_seq":26839,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.983657+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:44.983657+0000","last_peered":"2024-09-16T04:58:44.983657+0000","last_clean":"2024-09-16T04:58:44.983657+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:44.983657+0000","last_undegraded":"2024-09-16T04:58:44.983657+0000","last_fullsized":"2024-09-16T04:58:44.983657+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24576,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16695,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19863","reported_seq":19972,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:44.944256+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:44.944256+0000","last_peered":"2024-09-16T04:58:44.944256+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:44.944256+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53865,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19862,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3393}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20527","reported_seq":20641,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:44.972115+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:44.972115+0000","last_peered":"2024-09-16T04:58:44.972115+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:44.972115+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22331,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20524,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3506}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20186","reported_seq":26991,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.797433+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:44.797433+0000","last_peered":"2024-09-16T04:58:44.797433+0000","last_clean":"2024-09-16T04:58:44.797433+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:44.797433+0000","last_undegraded":"2024-09-16T04:58:44.797433+0000","last_fullsized":"2024-09-16T04:58:44.797433+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49152,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16940,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20405","reported_seq":20555,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.075425+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:45.075425+0000","last_peered":"2024-09-16T04:58:45.075425+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:45.075425+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45673,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17174,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":532727,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286493,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12310,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167972864,"data_stored":167898619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160715,"ondisk_log_size":160715,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493812,"kb_used_data":83092,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878028,"statfs":{"total":96636764160,"available":96131100672,"internally_reserved":0,"allocated":85086208,"data_stored":84792561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593838,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467240,"kb_used_data":2392,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93904600,"statfs":{"total":96636764160,"available":96158310400,"internally_reserved":0,"allocated":2449408,"data_stored":2100097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488708,"kb_used_data":82224,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93883132,"statfs":{"total":96636764160,"available":96136327168,"internally_reserved":0,"allocated":84197376,"data_stored":83907801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":83021824,"data_stored":83018345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":604433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84283392,"data_stored":84275841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:46.635 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:46.970 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:46.987 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-16T04:58:46.541612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":532746,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286495,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12311,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1449760,"kb_used_data":167708,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281665760,"statfs":{"total":289910292480,"available":288425738240,"internally_reserved":0,"allocated":171732992,"data_stored":170800459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,8,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29335552,"num_objects":-7162,"num_object_clones":0,"num_object_copies":-14324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7162,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004282"},"pg_stats":[{"pgid":"2.f","version":"51'19734","reported_seq":24485,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.086093+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:45.086093+0000","last_peered":"2024-09-16T04:58:45.086093+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:45.086093+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16384,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1011,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19734,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2331,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1007}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19985","reported_seq":20112,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.900860+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:44.900860+0000","last_peered":"2024-09-16T04:58:44.900860+0000","last_clean":"2024-09-16T04:58:44.900860+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:44.900860+0000","last_undegraded":"2024-09-16T04:58:44.900860+0000","last_fullsized":"2024-09-16T04:58:44.900860+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36864,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16954,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19851","reported_seq":20001,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:44.859402+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:44.859402+0000","last_peered":"2024-09-16T04:58:44.859402+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:44.859402+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24600,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16590,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20331","reported_seq":20411,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.989593+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:44.989593+0000","last_peered":"2024-09-16T04:58:44.989593+0000","last_clean":"2024-09-16T04:58:44.989593+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:44.989593+0000","last_undegraded":"2024-09-16T04:58:44.989593+0000","last_fullsized":"2024-09-16T04:58:44.989593+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53248,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20331,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20142","reported_seq":20372,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.071041+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:45.071041+0000","last_peered":"2024-09-16T04:58:45.071041+0000","last_clean":"2024-09-16T04:58:45.071041+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:45.071041+0000","last_undegraded":"2024-09-16T04:58:45.071041+0000","last_fullsized":"2024-09-16T04:58:45.071041+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8192,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17174,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20304","reported_seq":20448,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.011657+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:45.011657+0000","last_peered":"2024-09-16T04:58:45.011657+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:45.011657+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36864,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17297,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3447}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19851","reported_seq":20010,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.022432+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:45.022432+0000","last_peered":"2024-09-16T04:58:45.022432+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:45.022432+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36864,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16506,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3390}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20012","reported_seq":20239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.986469+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:44.986469+0000","last_peered":"2024-09-16T04:58:44.986469+0000","last_clean":"2024-09-16T04:58:44.986469+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:44.986469+0000","last_undegraded":"2024-09-16T04:58:44.986469+0000","last_fullsized":"2024-09-16T04:58:44.986469+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16384,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17082,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20073","reported_seq":20312,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:44.913407+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:44.913407+0000","last_peered":"2024-09-16T04:58:44.913407+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:44.913407+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32768,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16824,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3313}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20206","reported_seq":20346,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.039215+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:45.039215+0000","last_peered":"2024-09-16T04:58:45.039215+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:45.039215+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49152,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16861,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3453}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19973","reported_seq":26829,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.974985+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:44.974985+0000","last_peered":"2024-09-16T04:58:44.974985+0000","last_clean":"2024-09-16T04:58:44.974985+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:44.974985+0000","last_undegraded":"2024-09-16T04:58:44.974985+0000","last_fullsized":"2024-09-16T04:58:44.974985+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:45.744684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25810,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19945,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19972","reported_seq":26839,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.983657+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:44.983657+0000","last_peered":"2024-09-16T04:58:44.983657+0000","last_clean":"2024-09-16T04:58:44.983657+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:44.983657+0000","last_undegraded":"2024-09-16T04:58:44.983657+0000","last_fullsized":"2024-09-16T04:58:44.983657+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24576,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16695,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19863","reported_seq":19972,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:44.944256+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:44.944256+0000","last_peered":"2024-09-16T04:58:44.944256+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:44.944256+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53865,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19862,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3393}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20527","reported_seq":20641,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:44.972115+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:44.972115+0000","last_peered":"2024-09-16T04:58:44.972115+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:44.972115+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22331,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20524,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3506}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20186","reported_seq":26991,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.797433+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:44.797433+0000","last_peered":"2024-09-16T04:58:44.797433+0000","last_clean":"2024-09-16T04:58:44.797433+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:44.797433+0000","last_undegraded":"2024-09-16T04:58:44.797433+0000","last_fullsized":"2024-09-16T04:58:44.797433+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49152,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16940,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20405","reported_seq":20555,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.075425+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:45.075425+0000","last_peered":"2024-09-16T04:58:45.075425+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:45.075425+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45673,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17174,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":532727,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286493,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12310,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167972864,"data_stored":167898619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160715,"ondisk_log_size":160715,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493812,"kb_used_data":83092,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878028,"statfs":{"total":96636764160,"available":96131100672,"internally_reserved":0,"allocated":85086208,"data_stored":84792561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593838,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467240,"kb_used_data":2392,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93904600,"statfs":{"total":96636764160,"available":96158310400,"internally_reserved":0,"allocated":2449408,"data_stored":2100097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488708,"kb_used_data":82224,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93883132,"statfs":{"total":96636764160,"available":96136327168,"internally_reserved":0,"allocated":84197376,"data_stored":83907801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":83021824,"data_stored":83018345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":604433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84283392,"data_stored":84275841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:46.988 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:58:46.988 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'19734", 'reported_seq': 24485, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.086093+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:58:45.086093+0000', 'last_peered': '2024-09-16T04:58:45.086093+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:58:45.086093+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'9700", 'ondisk_log_start': "47'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16384, 'num_objects': 4, 'num_object_clones': 0, 'num_object_copies': 8, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1011, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19734, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2331, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1007}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:46.988 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:58:46.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19851", 'reported_seq': 20001, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:44.859402+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:44.859402+0000', 'last_peered': '2024-09-16T04:58:44.859402+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:44.859402+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24600, 'num_objects': 7, 'num_object_clones': 0, 'num_object_copies': 14, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16590, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3312}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:46.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:58:46.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20304", 'reported_seq': 20448, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.011657+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:45.011657+0000', 'last_peered': '2024-09-16T04:58:45.011657+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:45.011657+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36864, 'num_objects': 9, 'num_object_clones': 0, 'num_object_copies': 18, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17297, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3447}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:46.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:58:46.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19851", 'reported_seq': 20010, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.022432+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:45.022432+0000', 'last_peered': '2024-09-16T04:58:45.022432+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:45.022432+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36864, 'num_objects': 9, 'num_object_clones': 0, 'num_object_copies': 18, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16506, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3390}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:46.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:58:46.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20073", 'reported_seq': 20312, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:44.913407+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:44.913407+0000', 'last_peered': '2024-09-16T04:58:44.913407+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:44.913407+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32768, 'num_objects': 8, 'num_object_clones': 0, 'num_object_copies': 16, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16824, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3313}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:46.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:58:46.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20206", 'reported_seq': 20346, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.039215+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:45.039215+0000', 'last_peered': '2024-09-16T04:58:45.039215+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:45.039215+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 49152, 'num_objects': 12, 'num_object_clones': 0, 'num_object_copies': 24, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 12, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16861, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3453}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:46.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:58:46.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19863", 'reported_seq': 19972, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:44.944256+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:44.944256+0000', 'last_peered': '2024-09-16T04:58:44.944256+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:44.944256+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 53865, 'num_objects': 14, 'num_object_clones': 0, 'num_object_copies': 28, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 14, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19862, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3393}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:46.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:58:46.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'20527", 'reported_seq': 20641, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:44.972115+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:58:44.972115+0000', 'last_peered': '2024-09-16T04:58:44.972115+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:58:44.972115+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'10500", 'ondisk_log_start': "47'10500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22331, 'num_objects': 8, 'num_object_clones': 0, 'num_object_copies': 16, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20524, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3506}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:46.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:58:46.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20405", 'reported_seq': 20555, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.075425+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:45.075425+0000', 'last_peered': '2024-09-16T04:58:45.075425+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:45.075425+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 45673, 'num_objects': 12, 'num_object_clones': 0, 'num_object_copies': 24, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 12, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17174, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3512}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:46.990 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:58:47.411 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:47.411 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28432/278 objects degraded (10227.338%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":675,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":139,"data_bytes":532746,"bytes_used":1484554240,"bytes_avail":288425738240,"bytes_total":289910292480,"degraded_objects":28432,"degraded_total":278,"degraded_ratio":102.27338129496403,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":302,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":596},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:58:47.431 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:58:47.431 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:47.807 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:47.808 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:47.827 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-16T04:58:46.541612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":532746,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286495,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12311,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1449760,"kb_used_data":167708,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281665760,"statfs":{"total":289910292480,"available":288425738240,"internally_reserved":0,"allocated":171732992,"data_stored":170800459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,8,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29335552,"num_objects":-7162,"num_object_clones":0,"num_object_copies":-14324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7162,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004282"},"pg_stats":[{"pgid":"2.f","version":"51'19734","reported_seq":24485,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.086093+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:45.086093+0000","last_peered":"2024-09-16T04:58:45.086093+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:45.086093+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16384,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1011,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19734,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2331,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1007}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19985","reported_seq":20112,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.900860+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:44.900860+0000","last_peered":"2024-09-16T04:58:44.900860+0000","last_clean":"2024-09-16T04:58:44.900860+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:44.900860+0000","last_undegraded":"2024-09-16T04:58:44.900860+0000","last_fullsized":"2024-09-16T04:58:44.900860+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36864,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16954,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19851","reported_seq":20001,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:44.859402+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:44.859402+0000","last_peered":"2024-09-16T04:58:44.859402+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:44.859402+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24600,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16590,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20331","reported_seq":20411,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.989593+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:44.989593+0000","last_peered":"2024-09-16T04:58:44.989593+0000","last_clean":"2024-09-16T04:58:44.989593+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:44.989593+0000","last_undegraded":"2024-09-16T04:58:44.989593+0000","last_fullsized":"2024-09-16T04:58:44.989593+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53248,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20331,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20142","reported_seq":20372,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.071041+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:45.071041+0000","last_peered":"2024-09-16T04:58:45.071041+0000","last_clean":"2024-09-16T04:58:45.071041+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:45.071041+0000","last_undegraded":"2024-09-16T04:58:45.071041+0000","last_fullsized":"2024-09-16T04:58:45.071041+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8192,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17174,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20304","reported_seq":20448,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.011657+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:45.011657+0000","last_peered":"2024-09-16T04:58:45.011657+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:45.011657+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36864,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17297,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3447}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19851","reported_seq":20010,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.022432+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:45.022432+0000","last_peered":"2024-09-16T04:58:45.022432+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:45.022432+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36864,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16506,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3390}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20012","reported_seq":20239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.986469+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:44.986469+0000","last_peered":"2024-09-16T04:58:44.986469+0000","last_clean":"2024-09-16T04:58:44.986469+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:44.986469+0000","last_undegraded":"2024-09-16T04:58:44.986469+0000","last_fullsized":"2024-09-16T04:58:44.986469+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16384,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17082,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20073","reported_seq":20312,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:44.913407+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:44.913407+0000","last_peered":"2024-09-16T04:58:44.913407+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:44.913407+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32768,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16824,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3313}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20206","reported_seq":20346,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.039215+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:45.039215+0000","last_peered":"2024-09-16T04:58:45.039215+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:45.039215+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49152,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16861,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3453}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":109,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.713208+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:42.713208+0000","last_peered":"2024-09-16T04:56:42.713208+0000","last_clean":"2024-09-16T04:56:42.713208+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:42.713208+0000","last_undegraded":"2024-09-16T04:56:42.713208+0000","last_fullsized":"2024-09-16T04:56:42.713208+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19973","reported_seq":26829,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.974985+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:44.974985+0000","last_peered":"2024-09-16T04:58:44.974985+0000","last_clean":"2024-09-16T04:58:44.974985+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:44.974985+0000","last_undegraded":"2024-09-16T04:58:44.974985+0000","last_fullsized":"2024-09-16T04:58:44.974985+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:45.744684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25810,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19945,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19972","reported_seq":26839,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.983657+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:44.983657+0000","last_peered":"2024-09-16T04:58:44.983657+0000","last_clean":"2024-09-16T04:58:44.983657+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:44.983657+0000","last_undegraded":"2024-09-16T04:58:44.983657+0000","last_fullsized":"2024-09-16T04:58:44.983657+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24576,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16695,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19863","reported_seq":19972,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:44.944256+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:44.944256+0000","last_peered":"2024-09-16T04:58:44.944256+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:44.944256+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53865,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19862,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3393}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20527","reported_seq":20641,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:44.972115+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:44.972115+0000","last_peered":"2024-09-16T04:58:44.972115+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:44.972115+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22331,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20524,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3506}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20186","reported_seq":26991,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:44.797433+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:44.797433+0000","last_peered":"2024-09-16T04:58:44.797433+0000","last_clean":"2024-09-16T04:58:44.797433+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:44.797433+0000","last_undegraded":"2024-09-16T04:58:44.797433+0000","last_fullsized":"2024-09-16T04:58:44.797433+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49152,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16940,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20405","reported_seq":20555,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.075425+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:45.075425+0000","last_peered":"2024-09-16T04:58:45.075425+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:45.075425+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45673,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17174,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3512}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":532727,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286493,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12310,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167972864,"data_stored":167898619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160715,"ondisk_log_size":160715,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493812,"kb_used_data":83092,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878028,"statfs":{"total":96636764160,"available":96131100672,"internally_reserved":0,"allocated":85086208,"data_stored":84792561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593838,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467240,"kb_used_data":2392,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93904600,"statfs":{"total":96636764160,"available":96158310400,"internally_reserved":0,"allocated":2449408,"data_stored":2100097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488708,"kb_used_data":82224,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93883132,"statfs":{"total":96636764160,"available":96136327168,"internally_reserved":0,"allocated":84197376,"data_stored":83907801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":83021824,"data_stored":83018345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":604433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84283392,"data_stored":84275841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:50.830 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:51.170 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:51.170 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:51.189 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-16T04:58:50.543223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27871,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12872,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450360,"kb_used_data":164168,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665160,"statfs":{"total":289910292480,"available":288425123840,"internally_reserved":0,"allocated":168108032,"data_stored":167183643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20107288,"num_objects":-4910,"num_object_clones":0,"num_object_copies":-9820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4910,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":780,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004994"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":25611,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:50.085771+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:50.085771+0000","last_peered":"2024-09-16T04:58:50.085771+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:50.085771+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":450,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2892,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.233541+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:45.233541+0000","last_peered":"2024-09-16T04:58:45.233541+0000","last_clean":"2024-09-16T04:58:45.233541+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:45.233541+0000","last_undegraded":"2024-09-16T04:58:45.233541+0000","last_fullsized":"2024-09-16T04:58:45.233541+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20008,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.205937+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:45.205937+0000","last_peered":"2024-09-16T04:58:45.205937+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:45.205937+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20425,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.260969+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:45.260969+0000","last_peered":"2024-09-16T04:58:45.260969+0000","last_clean":"2024-09-16T04:58:45.260969+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:45.260969+0000","last_undegraded":"2024-09-16T04:58:45.260969+0000","last_fullsized":"2024-09-16T04:58:45.260969+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20374,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.165687+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:45.165687+0000","last_peered":"2024-09-16T04:58:45.165687+0000","last_clean":"2024-09-16T04:58:45.165687+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:45.165687+0000","last_undegraded":"2024-09-16T04:58:45.165687+0000","last_fullsized":"2024-09-16T04:58:45.165687+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20457,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.277775+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:45.277775+0000","last_peered":"2024-09-16T04:58:45.277775+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:45.277775+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20019,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.284386+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:45.284386+0000","last_peered":"2024-09-16T04:58:45.284386+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:45.284386+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20243,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.213365+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:45.213365+0000","last_peered":"2024-09-16T04:58:45.213365+0000","last_clean":"2024-09-16T04:58:45.213365+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:45.213365+0000","last_undegraded":"2024-09-16T04:58:45.213365+0000","last_fullsized":"2024-09-16T04:58:45.213365+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20320,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.188791+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:45.188791+0000","last_peered":"2024-09-16T04:58:45.188791+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:45.188791+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20359,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.277537+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:45.277537+0000","last_peered":"2024-09-16T04:58:45.277537+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:45.277537+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.546308+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:44.546308+0000","last_peered":"2024-09-16T04:56:44.546308+0000","last_clean":"2024-09-16T04:56:44.546308+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:44.546308+0000","last_undegraded":"2024-09-16T04:56:44.546308+0000","last_fullsized":"2024-09-16T04:56:44.546308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26835,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.166596+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:45.166596+0000","last_peered":"2024-09-16T04:58:45.166596+0000","last_clean":"2024-09-16T04:58:45.166596+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:45.166596+0000","last_undegraded":"2024-09-16T04:58:45.166596+0000","last_fullsized":"2024-09-16T04:58:45.166596+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:45.744684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26846,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.254605+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:45.254605+0000","last_peered":"2024-09-16T04:58:45.254605+0000","last_clean":"2024-09-16T04:58:45.254605+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:45.254605+0000","last_undegraded":"2024-09-16T04:58:45.254605+0000","last_fullsized":"2024-09-16T04:58:45.254605+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19985,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.268230+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:45.268230+0000","last_peered":"2024-09-16T04:58:45.268230+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:45.268230+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20646,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.115775+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:45.115775+0000","last_peered":"2024-09-16T04:58:45.115775+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:45.115775+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3511}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27003,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.220673+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:45.220673+0000","last_peered":"2024-09-16T04:58:45.220673+0000","last_clean":"2024-09-16T04:58:45.220673+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:45.220673+0000","last_undegraded":"2024-09-16T04:58:45.220673+0000","last_fullsized":"2024-09-16T04:58:45.220673+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20567,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.275844+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:45.275844+0000","last_peered":"2024-09-16T04:58:45.275844+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:45.275844+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27871,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12871,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166817792,"data_stored":166751691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493156,"kb_used_data":82452,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878684,"statfs":{"total":96636764160,"available":96131772416,"internally_reserved":0,"allocated":84430848,"data_stored":84141273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593839,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469496,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902344,"statfs":{"total":96636764160,"available":96156000256,"internally_reserved":0,"allocated":499712,"data_stored":154473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487708,"kb_used_data":81228,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884132,"statfs":{"total":96636764160,"available":96137351168,"internally_reserved":0,"allocated":83177472,"data_stored":82887897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":10489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:51.190 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:51.651 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:51.652 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:51.672 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-16T04:58:50.543223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27871,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12872,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450360,"kb_used_data":164168,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665160,"statfs":{"total":289910292480,"available":288425123840,"internally_reserved":0,"allocated":168108032,"data_stored":167183643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20107288,"num_objects":-4910,"num_object_clones":0,"num_object_copies":-9820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4910,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":780,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004994"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":25611,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:50.085771+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:50.085771+0000","last_peered":"2024-09-16T04:58:50.085771+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:50.085771+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":450,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2892,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.233541+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:45.233541+0000","last_peered":"2024-09-16T04:58:45.233541+0000","last_clean":"2024-09-16T04:58:45.233541+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:45.233541+0000","last_undegraded":"2024-09-16T04:58:45.233541+0000","last_fullsized":"2024-09-16T04:58:45.233541+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20008,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.205937+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:45.205937+0000","last_peered":"2024-09-16T04:58:45.205937+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:45.205937+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20425,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.260969+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:45.260969+0000","last_peered":"2024-09-16T04:58:45.260969+0000","last_clean":"2024-09-16T04:58:45.260969+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:45.260969+0000","last_undegraded":"2024-09-16T04:58:45.260969+0000","last_fullsized":"2024-09-16T04:58:45.260969+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20374,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.165687+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:45.165687+0000","last_peered":"2024-09-16T04:58:45.165687+0000","last_clean":"2024-09-16T04:58:45.165687+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:45.165687+0000","last_undegraded":"2024-09-16T04:58:45.165687+0000","last_fullsized":"2024-09-16T04:58:45.165687+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20457,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.277775+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:45.277775+0000","last_peered":"2024-09-16T04:58:45.277775+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:45.277775+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20019,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.284386+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:45.284386+0000","last_peered":"2024-09-16T04:58:45.284386+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:45.284386+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20243,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.213365+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:45.213365+0000","last_peered":"2024-09-16T04:58:45.213365+0000","last_clean":"2024-09-16T04:58:45.213365+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:45.213365+0000","last_undegraded":"2024-09-16T04:58:45.213365+0000","last_fullsized":"2024-09-16T04:58:45.213365+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20320,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.188791+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:45.188791+0000","last_peered":"2024-09-16T04:58:45.188791+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:45.188791+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20359,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.277537+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:45.277537+0000","last_peered":"2024-09-16T04:58:45.277537+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:45.277537+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.546308+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:44.546308+0000","last_peered":"2024-09-16T04:56:44.546308+0000","last_clean":"2024-09-16T04:56:44.546308+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:44.546308+0000","last_undegraded":"2024-09-16T04:56:44.546308+0000","last_fullsized":"2024-09-16T04:56:44.546308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26835,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.166596+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:45.166596+0000","last_peered":"2024-09-16T04:58:45.166596+0000","last_clean":"2024-09-16T04:58:45.166596+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:45.166596+0000","last_undegraded":"2024-09-16T04:58:45.166596+0000","last_fullsized":"2024-09-16T04:58:45.166596+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:45.744684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26846,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.254605+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:45.254605+0000","last_peered":"2024-09-16T04:58:45.254605+0000","last_clean":"2024-09-16T04:58:45.254605+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:45.254605+0000","last_undegraded":"2024-09-16T04:58:45.254605+0000","last_fullsized":"2024-09-16T04:58:45.254605+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19985,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.268230+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:45.268230+0000","last_peered":"2024-09-16T04:58:45.268230+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:45.268230+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20646,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.115775+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:45.115775+0000","last_peered":"2024-09-16T04:58:45.115775+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:45.115775+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3511}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27003,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.220673+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:45.220673+0000","last_peered":"2024-09-16T04:58:45.220673+0000","last_clean":"2024-09-16T04:58:45.220673+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:45.220673+0000","last_undegraded":"2024-09-16T04:58:45.220673+0000","last_fullsized":"2024-09-16T04:58:45.220673+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20567,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.275844+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:45.275844+0000","last_peered":"2024-09-16T04:58:45.275844+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:45.275844+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27871,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12871,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166817792,"data_stored":166751691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493156,"kb_used_data":82452,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878684,"statfs":{"total":96636764160,"available":96131772416,"internally_reserved":0,"allocated":84430848,"data_stored":84141273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593839,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469496,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902344,"statfs":{"total":96636764160,"available":96156000256,"internally_reserved":0,"allocated":499712,"data_stored":154473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487708,"kb_used_data":81228,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884132,"statfs":{"total":96636764160,"available":96137351168,"internally_reserved":0,"allocated":83177472,"data_stored":82887897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":10489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:51.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:58:51.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'19738", 'reported_seq': 25611, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:50.085771+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:58:50.085771+0000', 'last_peered': '2024-09-16T04:58:50.085771+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:58:50.085771+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'9700", 'ondisk_log_start': "47'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 450, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19738, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2892, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -450}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:51.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:58:51.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20008, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.205937+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:45.205937+0000', 'last_peered': '2024-09-16T04:58:45.205937+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:45.205937+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:51.673 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:58:51.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20457, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.277775+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:45.277775+0000', 'last_peered': '2024-09-16T04:58:45.277775+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:45.277775+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:51.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:58:51.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20019, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.284386+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:45.284386+0000', 'last_peered': '2024-09-16T04:58:45.284386+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:45.284386+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:51.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:58:51.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20320, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.188791+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:45.188791+0000', 'last_peered': '2024-09-16T04:58:45.188791+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:45.188791+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:51.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:58:51.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20359, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.277537+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:45.277537+0000', 'last_peered': '2024-09-16T04:58:45.277537+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:45.277537+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:51.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:58:51.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19985, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.268230+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:45.268230+0000', 'last_peered': '2024-09-16T04:58:45.268230+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:45.268230+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:51.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:58:51.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'20532", 'reported_seq': 20646, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.115775+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:58:45.115775+0000', 'last_peered': '2024-09-16T04:58:45.115775+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:58:45.115775+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'10500", 'ondisk_log_start': "47'10500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1851, 'num_objects': 3, 'num_object_clones': 0, 'num_object_copies': 6, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20529, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3511}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:51.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:58:51.675 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 20567, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.275844+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:45.275844+0000', 'last_peered': '2024-09-16T04:58:45.275844+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:45.275844+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:51.675 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:58:52.090 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:52.090 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27871/18 objects degraded (154838.889%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":679,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1485168640,"bytes_avail":288425123840,"bytes_total":289910292480,"degraded_objects":27871,"degraded_total":18,"degraded_ratio":1548.3888888888891,"recovering_objects_per_sec":64,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":780,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":408},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:58:52.111 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:58:52.111 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:52.429 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:52.430 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:52.448 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-16T04:58:50.543223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27871,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12872,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450360,"kb_used_data":164168,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665160,"statfs":{"total":289910292480,"available":288425123840,"internally_reserved":0,"allocated":168108032,"data_stored":167183643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20107288,"num_objects":-4910,"num_object_clones":0,"num_object_copies":-9820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4910,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":780,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004994"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":25611,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:50.085771+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:50.085771+0000","last_peered":"2024-09-16T04:58:50.085771+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:50.085771+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":450,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2892,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.233541+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:45.233541+0000","last_peered":"2024-09-16T04:58:45.233541+0000","last_clean":"2024-09-16T04:58:45.233541+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:45.233541+0000","last_undegraded":"2024-09-16T04:58:45.233541+0000","last_fullsized":"2024-09-16T04:58:45.233541+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20008,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.205937+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:45.205937+0000","last_peered":"2024-09-16T04:58:45.205937+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:45.205937+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20425,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.260969+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:45.260969+0000","last_peered":"2024-09-16T04:58:45.260969+0000","last_clean":"2024-09-16T04:58:45.260969+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:45.260969+0000","last_undegraded":"2024-09-16T04:58:45.260969+0000","last_fullsized":"2024-09-16T04:58:45.260969+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20374,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.165687+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:45.165687+0000","last_peered":"2024-09-16T04:58:45.165687+0000","last_clean":"2024-09-16T04:58:45.165687+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:45.165687+0000","last_undegraded":"2024-09-16T04:58:45.165687+0000","last_fullsized":"2024-09-16T04:58:45.165687+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20457,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.277775+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:45.277775+0000","last_peered":"2024-09-16T04:58:45.277775+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:45.277775+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20019,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.284386+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:45.284386+0000","last_peered":"2024-09-16T04:58:45.284386+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:45.284386+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20243,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.213365+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:45.213365+0000","last_peered":"2024-09-16T04:58:45.213365+0000","last_clean":"2024-09-16T04:58:45.213365+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:45.213365+0000","last_undegraded":"2024-09-16T04:58:45.213365+0000","last_fullsized":"2024-09-16T04:58:45.213365+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20320,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.188791+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:45.188791+0000","last_peered":"2024-09-16T04:58:45.188791+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:45.188791+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20359,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.277537+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:45.277537+0000","last_peered":"2024-09-16T04:58:45.277537+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:45.277537+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.546308+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:44.546308+0000","last_peered":"2024-09-16T04:56:44.546308+0000","last_clean":"2024-09-16T04:56:44.546308+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:44.546308+0000","last_undegraded":"2024-09-16T04:56:44.546308+0000","last_fullsized":"2024-09-16T04:56:44.546308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26835,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.166596+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:45.166596+0000","last_peered":"2024-09-16T04:58:45.166596+0000","last_clean":"2024-09-16T04:58:45.166596+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:45.166596+0000","last_undegraded":"2024-09-16T04:58:45.166596+0000","last_fullsized":"2024-09-16T04:58:45.166596+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:45.744684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26846,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.254605+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:45.254605+0000","last_peered":"2024-09-16T04:58:45.254605+0000","last_clean":"2024-09-16T04:58:45.254605+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:45.254605+0000","last_undegraded":"2024-09-16T04:58:45.254605+0000","last_fullsized":"2024-09-16T04:58:45.254605+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19985,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.268230+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:45.268230+0000","last_peered":"2024-09-16T04:58:45.268230+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:45.268230+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20646,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.115775+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:45.115775+0000","last_peered":"2024-09-16T04:58:45.115775+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:45.115775+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3511}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27003,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.220673+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:45.220673+0000","last_peered":"2024-09-16T04:58:45.220673+0000","last_clean":"2024-09-16T04:58:45.220673+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:45.220673+0000","last_undegraded":"2024-09-16T04:58:45.220673+0000","last_fullsized":"2024-09-16T04:58:45.220673+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20567,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.275844+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:45.275844+0000","last_peered":"2024-09-16T04:58:45.275844+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:45.275844+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27871,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12871,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166817792,"data_stored":166751691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493156,"kb_used_data":82452,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878684,"statfs":{"total":96636764160,"available":96131772416,"internally_reserved":0,"allocated":84430848,"data_stored":84141273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593839,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469496,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902344,"statfs":{"total":96636764160,"available":96156000256,"internally_reserved":0,"allocated":499712,"data_stored":154473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487708,"kb_used_data":81228,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884132,"statfs":{"total":96636764160,"available":96137351168,"internally_reserved":0,"allocated":83177472,"data_stored":82887897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":10489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:55.450 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:55.842 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:55.843 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:55.861 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-16T04:58:54.757548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27871,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12872,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450360,"kb_used_data":164168,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665160,"statfs":{"total":289910292480,"available":288425123840,"internally_reserved":0,"allocated":168108032,"data_stored":167183643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-528408,"num_objects":-130,"num_object_clones":0,"num_object_copies":-260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-561,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":130,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":561,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.215936"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":25611,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:50.085771+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:50.085771+0000","last_peered":"2024-09-16T04:58:50.085771+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:50.085771+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":450,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2892,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.233541+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:45.233541+0000","last_peered":"2024-09-16T04:58:45.233541+0000","last_clean":"2024-09-16T04:58:45.233541+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:45.233541+0000","last_undegraded":"2024-09-16T04:58:45.233541+0000","last_fullsized":"2024-09-16T04:58:45.233541+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20008,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.205937+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:45.205937+0000","last_peered":"2024-09-16T04:58:45.205937+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:45.205937+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20425,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.260969+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:45.260969+0000","last_peered":"2024-09-16T04:58:45.260969+0000","last_clean":"2024-09-16T04:58:45.260969+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:45.260969+0000","last_undegraded":"2024-09-16T04:58:45.260969+0000","last_fullsized":"2024-09-16T04:58:45.260969+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20374,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.165687+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:45.165687+0000","last_peered":"2024-09-16T04:58:45.165687+0000","last_clean":"2024-09-16T04:58:45.165687+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:45.165687+0000","last_undegraded":"2024-09-16T04:58:45.165687+0000","last_fullsized":"2024-09-16T04:58:45.165687+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20457,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.277775+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:45.277775+0000","last_peered":"2024-09-16T04:58:45.277775+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:45.277775+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20019,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.284386+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:45.284386+0000","last_peered":"2024-09-16T04:58:45.284386+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:45.284386+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20243,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.213365+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:45.213365+0000","last_peered":"2024-09-16T04:58:45.213365+0000","last_clean":"2024-09-16T04:58:45.213365+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:45.213365+0000","last_undegraded":"2024-09-16T04:58:45.213365+0000","last_fullsized":"2024-09-16T04:58:45.213365+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20320,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.188791+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:45.188791+0000","last_peered":"2024-09-16T04:58:45.188791+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:45.188791+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20359,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.277537+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:45.277537+0000","last_peered":"2024-09-16T04:58:45.277537+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:45.277537+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.546308+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:44.546308+0000","last_peered":"2024-09-16T04:56:44.546308+0000","last_clean":"2024-09-16T04:56:44.546308+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:44.546308+0000","last_undegraded":"2024-09-16T04:56:44.546308+0000","last_fullsized":"2024-09-16T04:56:44.546308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26835,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.166596+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:45.166596+0000","last_peered":"2024-09-16T04:58:45.166596+0000","last_clean":"2024-09-16T04:58:45.166596+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:45.166596+0000","last_undegraded":"2024-09-16T04:58:45.166596+0000","last_fullsized":"2024-09-16T04:58:45.166596+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:45.744684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26846,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.254605+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:45.254605+0000","last_peered":"2024-09-16T04:58:45.254605+0000","last_clean":"2024-09-16T04:58:45.254605+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:45.254605+0000","last_undegraded":"2024-09-16T04:58:45.254605+0000","last_fullsized":"2024-09-16T04:58:45.254605+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19985,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.268230+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:45.268230+0000","last_peered":"2024-09-16T04:58:45.268230+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:45.268230+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20646,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.115775+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:45.115775+0000","last_peered":"2024-09-16T04:58:45.115775+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:45.115775+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3511}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27003,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.220673+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:45.220673+0000","last_peered":"2024-09-16T04:58:45.220673+0000","last_clean":"2024-09-16T04:58:45.220673+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:45.220673+0000","last_undegraded":"2024-09-16T04:58:45.220673+0000","last_fullsized":"2024-09-16T04:58:45.220673+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20567,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.275844+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:45.275844+0000","last_peered":"2024-09-16T04:58:45.275844+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:45.275844+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27871,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12871,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166817792,"data_stored":166751691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493156,"kb_used_data":82452,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878684,"statfs":{"total":96636764160,"available":96131772416,"internally_reserved":0,"allocated":84430848,"data_stored":84141273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593839,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469496,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902344,"statfs":{"total":96636764160,"available":96156000256,"internally_reserved":0,"allocated":499712,"data_stored":154473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487708,"kb_used_data":81228,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884132,"statfs":{"total":96636764160,"available":96137351168,"internally_reserved":0,"allocated":83177472,"data_stored":82887897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":10489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:55.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:56.237 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:56.238 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:56.258 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-16T04:58:54.757548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27871,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12872,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450360,"kb_used_data":164168,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665160,"statfs":{"total":289910292480,"available":288425123840,"internally_reserved":0,"allocated":168108032,"data_stored":167183643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-528408,"num_objects":-130,"num_object_clones":0,"num_object_copies":-260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-561,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":130,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":561,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.215936"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":25611,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:50.085771+0000","last_change":"2024-09-16T04:56:41.544928+0000","last_active":"2024-09-16T04:58:50.085771+0000","last_peered":"2024-09-16T04:58:50.085771+0000","last_clean":"2024-09-16T04:49:35.616010+0000","last_became_active":"2024-09-16T04:49:38.190952+0000","last_became_peered":"2024-09-16T04:49:38.190952+0000","last_unstale":"2024-09-16T04:58:50.085771+0000","last_undegraded":"2024-09-16T04:49:37.762885+0000","last_fullsized":"2024-09-16T04:49:37.646023+0000","mapping_epoch":44,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":450,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2892,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20121,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.233541+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:45.233541+0000","last_peered":"2024-09-16T04:58:45.233541+0000","last_clean":"2024-09-16T04:58:45.233541+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:45.233541+0000","last_undegraded":"2024-09-16T04:58:45.233541+0000","last_fullsized":"2024-09-16T04:58:45.233541+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20008,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.205937+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:45.205937+0000","last_peered":"2024-09-16T04:58:45.205937+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:45.205937+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20425,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.260969+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:45.260969+0000","last_peered":"2024-09-16T04:58:45.260969+0000","last_clean":"2024-09-16T04:58:45.260969+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:45.260969+0000","last_undegraded":"2024-09-16T04:58:45.260969+0000","last_fullsized":"2024-09-16T04:58:45.260969+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20374,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.165687+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:45.165687+0000","last_peered":"2024-09-16T04:58:45.165687+0000","last_clean":"2024-09-16T04:58:45.165687+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:45.165687+0000","last_undegraded":"2024-09-16T04:58:45.165687+0000","last_fullsized":"2024-09-16T04:58:45.165687+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20457,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.277775+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:45.277775+0000","last_peered":"2024-09-16T04:58:45.277775+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:45.277775+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20019,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.284386+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:45.284386+0000","last_peered":"2024-09-16T04:58:45.284386+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:45.284386+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20243,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.213365+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:45.213365+0000","last_peered":"2024-09-16T04:58:45.213365+0000","last_clean":"2024-09-16T04:58:45.213365+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:45.213365+0000","last_undegraded":"2024-09-16T04:58:45.213365+0000","last_fullsized":"2024-09-16T04:58:45.213365+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710149+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:56:42.710149+0000","last_peered":"2024-09-16T04:56:42.710149+0000","last_clean":"2024-09-16T04:56:42.710149+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:56:42.710149+0000","last_undegraded":"2024-09-16T04:56:42.710149+0000","last_fullsized":"2024-09-16T04:56:42.710149+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20320,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.188791+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:45.188791+0000","last_peered":"2024-09-16T04:58:45.188791+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:45.188791+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715157+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:56:42.715157+0000","last_peered":"2024-09-16T04:56:42.715157+0000","last_clean":"2024-09-16T04:56:42.715157+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:56:42.715157+0000","last_undegraded":"2024-09-16T04:56:42.715157+0000","last_fullsized":"2024-09-16T04:56:42.715157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20359,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.277537+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:45.277537+0000","last_peered":"2024-09-16T04:58:45.277537+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:45.277537+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":110,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:44.546308+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:56:44.546308+0000","last_peered":"2024-09-16T04:56:44.546308+0000","last_clean":"2024-09-16T04:56:44.546308+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:56:44.546308+0000","last_undegraded":"2024-09-16T04:56:44.546308+0000","last_fullsized":"2024-09-16T04:56:44.546308+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26835,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.166596+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:45.166596+0000","last_peered":"2024-09-16T04:58:45.166596+0000","last_clean":"2024-09-16T04:58:45.166596+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:45.166596+0000","last_undegraded":"2024-09-16T04:58:45.166596+0000","last_fullsized":"2024-09-16T04:58:45.166596+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:45.744684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710461+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:56:42.710461+0000","last_peered":"2024-09-16T04:56:42.710461+0000","last_clean":"2024-09-16T04:56:42.710461+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:56:42.710461+0000","last_undegraded":"2024-09-16T04:56:42.710461+0000","last_fullsized":"2024-09-16T04:56:42.710461+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26846,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.254605+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:45.254605+0000","last_peered":"2024-09-16T04:58:45.254605+0000","last_clean":"2024-09-16T04:58:45.254605+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:45.254605+0000","last_undegraded":"2024-09-16T04:58:45.254605+0000","last_fullsized":"2024-09-16T04:58:45.254605+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19985,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.268230+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:45.268230+0000","last_peered":"2024-09-16T04:58:45.268230+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:45.268230+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.711177+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:56:42.711177+0000","last_peered":"2024-09-16T04:56:42.711177+0000","last_clean":"2024-09-16T04:56:42.711177+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:56:42.711177+0000","last_undegraded":"2024-09-16T04:56:42.711177+0000","last_fullsized":"2024-09-16T04:56:42.711177+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20646,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.115775+0000","last_change":"2024-09-16T04:49:38.175800+0000","last_active":"2024-09-16T04:58:45.115775+0000","last_peered":"2024-09-16T04:58:45.115775+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:45.115775+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3511}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.715450+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:56:42.715450+0000","last_peered":"2024-09-16T04:56:42.715450+0000","last_clean":"2024-09-16T04:56:42.715450+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:56:42.715450+0000","last_undegraded":"2024-09-16T04:56:42.715450+0000","last_fullsized":"2024-09-16T04:56:42.715450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27003,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:58:45.220673+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:45.220673+0000","last_peered":"2024-09-16T04:58:45.220673+0000","last_clean":"2024-09-16T04:58:45.220673+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:45.220673+0000","last_undegraded":"2024-09-16T04:58:45.220673+0000","last_fullsized":"2024-09-16T04:58:45.220673+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710588+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:56:42.710588+0000","last_peered":"2024-09-16T04:56:42.710588+0000","last_clean":"2024-09-16T04:56:42.710588+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:56:42.710588+0000","last_undegraded":"2024-09-16T04:56:42.710588+0000","last_fullsized":"2024-09-16T04:56:42.710588+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-16T04:56:42.710981+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:56:42.710981+0000","last_peered":"2024-09-16T04:56:42.710981+0000","last_clean":"2024-09-16T04:56:42.710981+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:56:42.710981+0000","last_undegraded":"2024-09-16T04:56:42.710981+0000","last_fullsized":"2024-09-16T04:56:42.710981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20567,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:45.275844+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:45.275844+0000","last_peered":"2024-09-16T04:58:45.275844+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:45.275844+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27871,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12871,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166817792,"data_stored":166751691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493156,"kb_used_data":82452,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878684,"statfs":{"total":96636764160,"available":96131772416,"internally_reserved":0,"allocated":84430848,"data_stored":84141273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593839,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469496,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902344,"statfs":{"total":96636764160,"available":96156000256,"internally_reserved":0,"allocated":499712,"data_stored":154473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487708,"kb_used_data":81228,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884132,"statfs":{"total":96636764160,"available":96137351168,"internally_reserved":0,"allocated":83177472,"data_stored":82887897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":10489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:58:56.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-16T04:58:56.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "51'19738", 'reported_seq': 25611, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:50.085771+0000', 'last_change': '2024-09-16T04:56:41.544928+0000', 'last_active': '2024-09-16T04:58:50.085771+0000', 'last_peered': '2024-09-16T04:58:50.085771+0000', 'last_clean': '2024-09-16T04:49:35.616010+0000', 'last_became_active': '2024-09-16T04:49:38.190952+0000', 'last_became_peered': '2024-09-16T04:49:38.190952+0000', 'last_unstale': '2024-09-16T04:58:50.085771+0000', 'last_undegraded': '2024-09-16T04:49:37.762885+0000', 'last_fullsized': '2024-09-16T04:49:37.646023+0000', 'mapping_epoch': 44, 'log_start': "47'9700", 'ondisk_log_start': "47'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 450, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19738, 'num_write_kb': 39476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2892, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -450}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:56.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:58:56.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20008, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.205937+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:45.205937+0000', 'last_peered': '2024-09-16T04:58:45.205937+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:45.205937+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:56.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:58:56.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20457, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.277775+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:45.277775+0000', 'last_peered': '2024-09-16T04:58:45.277775+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:45.277775+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:56.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:58:56.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20019, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.284386+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:45.284386+0000', 'last_peered': '2024-09-16T04:58:45.284386+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:45.284386+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:56.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:58:56.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20320, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.188791+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:45.188791+0000', 'last_peered': '2024-09-16T04:58:45.188791+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:45.188791+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:56.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:58:56.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20359, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.277537+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:45.277537+0000', 'last_peered': '2024-09-16T04:58:45.277537+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:45.277537+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:56.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:58:56.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19985, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.268230+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:45.268230+0000', 'last_peered': '2024-09-16T04:58:45.268230+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:45.268230+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:56.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:58:56.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'20532", 'reported_seq': 20646, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.115775+0000', 'last_change': '2024-09-16T04:49:38.175800+0000', 'last_active': '2024-09-16T04:58:45.115775+0000', 'last_peered': '2024-09-16T04:58:45.115775+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:58:45.115775+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'10500", 'ondisk_log_start': "47'10500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1851, 'num_objects': 3, 'num_object_clones': 0, 'num_object_copies': 6, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20529, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3511}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:58:56.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:58:56.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 20567, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:45.275844+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:45.275844+0000', 'last_peered': '2024-09-16T04:58:45.275844+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:45.275844+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:58:56.261 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:58:56.659 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:56.660 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27871/18 objects degraded (154838.889%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":684,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1485168640,"bytes_avail":288425123840,"bytes_total":289910292480,"degraded_objects":27871,"degraded_total":18,"degraded_ratio":1548.3888888888891,"recovering_objects_per_sec":67,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":672,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":251},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:58:56.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:58:56.681 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:58:57.033 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:58:57.033 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:58:57.056 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-09-16T04:58:56.545311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13484,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454556,"kb_used_data":164188,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660964,"statfs":{"total":289910292480,"available":288420827136,"internally_reserved":0,"allocated":168128512,"data_stored":167193839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-528408,"num_objects":-130,"num_object_clones":0,"num_object_copies":-260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":130,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1173,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003048"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758689+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758689+0000","last_peered":"2024-09-16T04:58:54.758689+0000","last_clean":"2024-09-16T04:58:54.758689+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758689+0000","last_undegraded":"2024-09-16T04:58:54.758689+0000","last_fullsized":"2024-09-16T04:58:54.758689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20975,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.999748+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:58:54.999748+0000","last_peered":"2024-09-16T04:58:54.999748+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:54.999748+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3352}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760873+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760873+0000","last_peered":"2024-09-16T04:58:54.760873+0000","last_clean":"2024-09-16T04:58:54.760873+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760873+0000","last_undegraded":"2024-09-16T04:58:54.760873+0000","last_fullsized":"2024-09-16T04:58:54.760873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13483,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493168,"kb_used_data":82460,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878672,"statfs":{"total":96636764160,"available":96131760128,"internally_reserved":0,"allocated":84439040,"data_stored":84144290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593840,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473668,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898172,"statfs":{"total":96636764160,"available":96151728128,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487720,"kb_used_data":81236,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884120,"statfs":{"total":96636764160,"available":96137338880,"internally_reserved":0,"allocated":83185664,"data_stored":82890914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:00.058 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:00.466 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:00.467 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:00.490 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-16T04:58:58.545712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13484,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454556,"kb_used_data":164188,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660964,"statfs":{"total":289910292480,"available":288420827136,"internally_reserved":0,"allocated":168128512,"data_stored":167193839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-612,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":612,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002489"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758689+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758689+0000","last_peered":"2024-09-16T04:58:54.758689+0000","last_clean":"2024-09-16T04:58:54.758689+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758689+0000","last_undegraded":"2024-09-16T04:58:54.758689+0000","last_fullsized":"2024-09-16T04:58:54.758689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20975,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.999748+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:58:54.999748+0000","last_peered":"2024-09-16T04:58:54.999748+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:58:54.999748+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3352}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760873+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760873+0000","last_peered":"2024-09-16T04:58:54.760873+0000","last_clean":"2024-09-16T04:58:54.760873+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760873+0000","last_undegraded":"2024-09-16T04:58:54.760873+0000","last_fullsized":"2024-09-16T04:58:54.760873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13483,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493168,"kb_used_data":82460,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878672,"statfs":{"total":96636764160,"available":96131760128,"internally_reserved":0,"allocated":84439040,"data_stored":84144290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593840,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473668,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898172,"statfs":{"total":96636764160,"available":96151728128,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487720,"kb_used_data":81236,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884120,"statfs":{"total":96636764160,"available":96137338880,"internally_reserved":0,"allocated":83185664,"data_stored":82890914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:00.492 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:01.001 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:01.020 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-16T04:59:00.546203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14136,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442092,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673428,"statfs":{"total":289910292480,"available":288433590272,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1264,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002173"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758689+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758689+0000","last_peered":"2024-09-16T04:58:54.758689+0000","last_clean":"2024-09-16T04:58:54.758689+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758689+0000","last_undegraded":"2024-09-16T04:58:54.758689+0000","last_fullsized":"2024-09-16T04:58:54.758689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":22279,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:00.013442+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:00.013442+0000","last_peered":"2024-09-16T04:59:00.013442+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:00.013442+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2703,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":818,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2700}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760873+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760873+0000","last_peered":"2024-09-16T04:58:54.760873+0000","last_clean":"2024-09-16T04:58:54.760873+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760873+0000","last_undegraded":"2024-09-16T04:58:54.760873+0000","last_fullsized":"2024-09-16T04:58:54.760873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14135,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593841,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461180,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910660,"statfs":{"total":96636764160,"available":96164515840,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:01.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:01.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20012, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.759098+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:54.759098+0000', 'last_peered': '2024-09-16T04:58:54.759098+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:54.759098+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:01.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:01.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20461, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766815+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:54.766815+0000', 'last_peered': '2024-09-16T04:58:54.766815+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:54.766815+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:01.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:01.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20023, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766509+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:54.766509+0000', 'last_peered': '2024-09-16T04:58:54.766509+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:54.766509+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:01.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:01.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20324, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.759908+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:54.759908+0000', 'last_peered': '2024-09-16T04:58:54.759908+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:54.759908+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:01.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:01.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20363, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.768869+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:54.768869+0000', 'last_peered': '2024-09-16T04:58:54.768869+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:54.768869+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:01.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:01.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19989, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.760339+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:54.760339+0000', 'last_peered': '2024-09-16T04:58:54.760339+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:54.760339+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:01.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:59:01.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'20532", 'reported_seq': 22279, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:00.013442+0000', 'last_change': '2024-09-16T04:58:53.366597+0000', 'last_active': '2024-09-16T04:59:00.013442+0000', 'last_peered': '2024-09-16T04:59:00.013442+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:59:00.013442+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'10500", 'ondisk_log_start': "47'10500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1851, 'num_objects': 3, 'num_object_clones': 0, 'num_object_copies': 6, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2703, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20529, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 818, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2700}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:01.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:01.023 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 20571, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766610+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:54.766610+0000', 'last_peered': '2024-09-16T04:58:54.766610+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:54.766610+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:01.023 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:01.495 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:01.495 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26607/18 objects degraded (147816.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":689,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1476702208,"bytes_avail":288433590272,"bytes_total":289910292480,"degraded_objects":26607,"degraded_total":18,"degraded_ratio":1478.166666666667,"recovering_objects_per_sec":157,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1264,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:59:01.517 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:59:01.517 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:02.049 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:02.050 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:02.071 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-16T04:59:00.546203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14136,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442092,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673428,"statfs":{"total":289910292480,"available":288433590272,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1264,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002173"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758689+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758689+0000","last_peered":"2024-09-16T04:58:54.758689+0000","last_clean":"2024-09-16T04:58:54.758689+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758689+0000","last_undegraded":"2024-09-16T04:58:54.758689+0000","last_fullsized":"2024-09-16T04:58:54.758689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":22279,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:00.013442+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:00.013442+0000","last_peered":"2024-09-16T04:59:00.013442+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:00.013442+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2703,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":818,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2700}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760873+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760873+0000","last_peered":"2024-09-16T04:58:54.760873+0000","last_clean":"2024-09-16T04:58:54.760873+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760873+0000","last_undegraded":"2024-09-16T04:58:54.760873+0000","last_fullsized":"2024-09-16T04:58:54.760873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14135,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593841,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461180,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910660,"statfs":{"total":96636764160,"available":96164515840,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:05.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:05.429 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:05.448 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-16T04:59:04.547159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14136,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442092,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673428,"statfs":{"total":289910292480,"available":288433590272,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1264,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002654"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758689+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758689+0000","last_peered":"2024-09-16T04:58:54.758689+0000","last_clean":"2024-09-16T04:58:54.758689+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758689+0000","last_undegraded":"2024-09-16T04:58:54.758689+0000","last_fullsized":"2024-09-16T04:58:54.758689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":22279,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:00.013442+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:00.013442+0000","last_peered":"2024-09-16T04:59:00.013442+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:00.013442+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2703,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":818,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2700}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760873+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760873+0000","last_peered":"2024-09-16T04:58:54.760873+0000","last_clean":"2024-09-16T04:58:54.760873+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760873+0000","last_undegraded":"2024-09-16T04:58:54.760873+0000","last_fullsized":"2024-09-16T04:58:54.760873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14135,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593841,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461180,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910660,"statfs":{"total":96636764160,"available":96164515840,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:05.450 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:05.765 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:05.765 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:05.784 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-16T04:59:04.547159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14136,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442092,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673428,"statfs":{"total":289910292480,"available":288433590272,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1264,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002654"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758689+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758689+0000","last_peered":"2024-09-16T04:58:54.758689+0000","last_clean":"2024-09-16T04:58:54.758689+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758689+0000","last_undegraded":"2024-09-16T04:58:54.758689+0000","last_fullsized":"2024-09-16T04:58:54.758689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":22279,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:00.013442+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:00.013442+0000","last_peered":"2024-09-16T04:59:00.013442+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:00.013442+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2703,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":818,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2700}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760873+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760873+0000","last_peered":"2024-09-16T04:58:54.760873+0000","last_clean":"2024-09-16T04:58:54.760873+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760873+0000","last_undegraded":"2024-09-16T04:58:54.760873+0000","last_fullsized":"2024-09-16T04:58:54.760873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14135,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593841,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461180,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910660,"statfs":{"total":96636764160,"available":96164515840,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:05.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:05.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20012, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.759098+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:54.759098+0000', 'last_peered': '2024-09-16T04:58:54.759098+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:54.759098+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:05.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:05.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20461, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766815+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:54.766815+0000', 'last_peered': '2024-09-16T04:58:54.766815+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:54.766815+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:05.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:05.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20023, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766509+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:54.766509+0000', 'last_peered': '2024-09-16T04:58:54.766509+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:54.766509+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:05.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:05.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20324, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.759908+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:54.759908+0000', 'last_peered': '2024-09-16T04:58:54.759908+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:54.759908+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:05.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:05.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20363, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.768869+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:54.768869+0000', 'last_peered': '2024-09-16T04:58:54.768869+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:54.768869+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:05.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:05.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19989, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.760339+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:54.760339+0000', 'last_peered': '2024-09-16T04:58:54.760339+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:54.760339+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:05.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:59:05.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'20532", 'reported_seq': 22279, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:00.013442+0000', 'last_change': '2024-09-16T04:58:53.366597+0000', 'last_active': '2024-09-16T04:59:00.013442+0000', 'last_peered': '2024-09-16T04:59:00.013442+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:59:00.013442+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'10500", 'ondisk_log_start': "47'10500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1851, 'num_objects': 3, 'num_object_clones': 0, 'num_object_copies': 6, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2703, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20529, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 818, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2700}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:05.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:05.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 20571, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766610+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:54.766610+0000', 'last_peered': '2024-09-16T04:58:54.766610+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:54.766610+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:05.788 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:06.207 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:06.207 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26607/18 objects degraded (147816.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":693,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1476702208,"bytes_avail":288433590272,"bytes_total":289910292480,"degraded_objects":26607,"degraded_total":18,"degraded_ratio":1478.166666666667,"recovering_objects_per_sec":126,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1264,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:59:06.228 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:59:06.228 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:06.539 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:06.558 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-16T04:59:04.547159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14136,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442092,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673428,"statfs":{"total":289910292480,"available":288433590272,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1264,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002654"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758689+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758689+0000","last_peered":"2024-09-16T04:58:54.758689+0000","last_clean":"2024-09-16T04:58:54.758689+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758689+0000","last_undegraded":"2024-09-16T04:58:54.758689+0000","last_fullsized":"2024-09-16T04:58:54.758689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":22279,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:00.013442+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:00.013442+0000","last_peered":"2024-09-16T04:59:00.013442+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:00.013442+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2703,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":818,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2700}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760873+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760873+0000","last_peered":"2024-09-16T04:58:54.760873+0000","last_clean":"2024-09-16T04:58:54.760873+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760873+0000","last_undegraded":"2024-09-16T04:58:54.760873+0000","last_fullsized":"2024-09-16T04:58:54.760873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14135,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593841,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461180,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910660,"statfs":{"total":96636764160,"available":96164515840,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:09.560 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:09.890 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:09.890 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:09.911 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-16T04:59:08.548444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14881,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450412,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665108,"statfs":{"total":289910292480,"available":288425070592,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1397,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003132"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758689+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758689+0000","last_peered":"2024-09-16T04:58:54.758689+0000","last_clean":"2024-09-16T04:58:54.758689+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758689+0000","last_undegraded":"2024-09-16T04:58:54.758689+0000","last_fullsized":"2024-09-16T04:58:54.758689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":23769,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:05.020992+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:05.020992+0000","last_peered":"2024-09-16T04:59:05.020992+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:05.020992+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1563,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1955}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760873+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760873+0000","last_peered":"2024-09-16T04:58:54.760873+0000","last_clean":"2024-09-16T04:58:54.760873+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760873+0000","last_undegraded":"2024-09-16T04:58:54.760873+0000","last_fullsized":"2024-09-16T04:58:54.760873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14880,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593842,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469500,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902340,"statfs":{"total":96636764160,"available":96155996160,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:09.912 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:10.215 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:10.216 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:10.234 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-16T04:59:08.548444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14881,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450412,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665108,"statfs":{"total":289910292480,"available":288425070592,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1397,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003132"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758689+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758689+0000","last_peered":"2024-09-16T04:58:54.758689+0000","last_clean":"2024-09-16T04:58:54.758689+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758689+0000","last_undegraded":"2024-09-16T04:58:54.758689+0000","last_fullsized":"2024-09-16T04:58:54.758689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":23769,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:05.020992+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:05.020992+0000","last_peered":"2024-09-16T04:59:05.020992+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:05.020992+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1958,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1563,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1955}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760873+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760873+0000","last_peered":"2024-09-16T04:58:54.760873+0000","last_clean":"2024-09-16T04:58:54.760873+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760873+0000","last_undegraded":"2024-09-16T04:58:54.760873+0000","last_fullsized":"2024-09-16T04:58:54.760873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14880,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593842,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469500,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902340,"statfs":{"total":96636764160,"available":96155996160,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:10.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:10.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20012, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.759098+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:54.759098+0000', 'last_peered': '2024-09-16T04:58:54.759098+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:54.759098+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:10.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:10.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20461, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766815+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:54.766815+0000', 'last_peered': '2024-09-16T04:58:54.766815+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:54.766815+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:10.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:10.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20023, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766509+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:54.766509+0000', 'last_peered': '2024-09-16T04:58:54.766509+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:54.766509+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:10.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:10.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20324, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.759908+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:54.759908+0000', 'last_peered': '2024-09-16T04:58:54.759908+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:54.759908+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:10.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:10.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20363, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.768869+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:54.768869+0000', 'last_peered': '2024-09-16T04:58:54.768869+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:54.768869+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:10.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:10.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19989, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.760339+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:54.760339+0000', 'last_peered': '2024-09-16T04:58:54.760339+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:54.760339+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:10.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:59:10.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'20532", 'reported_seq': 23769, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:05.020992+0000', 'last_change': '2024-09-16T04:58:53.366597+0000', 'last_active': '2024-09-16T04:59:05.020992+0000', 'last_peered': '2024-09-16T04:59:05.020992+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:59:05.020992+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'10500", 'ondisk_log_start': "47'10500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1851, 'num_objects': 3, 'num_object_clones': 0, 'num_object_copies': 6, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1958, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20529, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1563, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1955}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:10.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:10.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 20571, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766610+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:54.766610+0000', 'last_peered': '2024-09-16T04:58:54.766610+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:54.766610+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:10.238 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:10.616 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:10.616 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25862/18 objects degraded (143677.778%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":698,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1485221888,"bytes_avail":288425070592,"bytes_total":289910292480,"degraded_objects":25862,"degraded_total":18,"degraded_ratio":1436.7777777777778,"recovering_objects_per_sec":116,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1397,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:59:10.636 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:59:10.637 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:10.954 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:10.955 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:10.973 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-16T04:59:10.549229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25437,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15306,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454572,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660948,"statfs":{"total":289910292480,"available":288420810752,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1822,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1822,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003516"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758816+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758816+0000","last_peered":"2024-09-16T04:58:54.758816+0000","last_clean":"2024-09-16T04:58:54.758816+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758816+0000","last_undegraded":"2024-09-16T04:58:54.758816+0000","last_fullsized":"2024-09-16T04:58:54.758816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":24619,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:10.021098+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:10.021098+0000","last_peered":"2024-09-16T04:59:10.021098+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:10.021098+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1988,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1530}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760873+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760873+0000","last_peered":"2024-09-16T04:58:54.760873+0000","last_clean":"2024-09-16T04:58:54.760873+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760873+0000","last_undegraded":"2024-09-16T04:58:54.760873+0000","last_fullsized":"2024-09-16T04:58:54.760873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25437,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15305,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593843,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:13.976 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:14.296 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:14.314 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-16T04:59:12.549999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25437,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15306,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454572,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660948,"statfs":{"total":289910292480,"available":288420810752,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1170,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003795"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758816+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758816+0000","last_peered":"2024-09-16T04:58:54.758816+0000","last_clean":"2024-09-16T04:58:54.758816+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758816+0000","last_undegraded":"2024-09-16T04:58:54.758816+0000","last_fullsized":"2024-09-16T04:58:54.758816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":24619,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:10.021098+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:10.021098+0000","last_peered":"2024-09-16T04:59:10.021098+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:10.021098+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1988,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1530}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760873+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760873+0000","last_peered":"2024-09-16T04:58:54.760873+0000","last_clean":"2024-09-16T04:58:54.760873+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760873+0000","last_undegraded":"2024-09-16T04:58:54.760873+0000","last_fullsized":"2024-09-16T04:58:54.760873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25437,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15305,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593843,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:14.315 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:14.828 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:14.829 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:14.847 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-16T04:59:14.550657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25437,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15306,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454572,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660948,"statfs":{"total":289910292480,"available":288420810752,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1170,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003977"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758816+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758816+0000","last_peered":"2024-09-16T04:58:54.758816+0000","last_clean":"2024-09-16T04:58:54.758816+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758816+0000","last_undegraded":"2024-09-16T04:58:54.758816+0000","last_fullsized":"2024-09-16T04:58:54.758816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":24619,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:10.021098+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:10.021098+0000","last_peered":"2024-09-16T04:59:10.021098+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:10.021098+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1988,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1530}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760873+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760873+0000","last_peered":"2024-09-16T04:58:54.760873+0000","last_clean":"2024-09-16T04:58:54.760873+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760873+0000","last_undegraded":"2024-09-16T04:58:54.760873+0000","last_fullsized":"2024-09-16T04:58:54.760873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25437,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15305,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593843,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20012, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.759098+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:54.759098+0000', 'last_peered': '2024-09-16T04:58:54.759098+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:54.759098+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20461, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766815+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:54.766815+0000', 'last_peered': '2024-09-16T04:58:54.766815+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:54.766815+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20023, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766509+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:54.766509+0000', 'last_peered': '2024-09-16T04:58:54.766509+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:54.766509+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:14.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20324, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.759908+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:54.759908+0000', 'last_peered': '2024-09-16T04:58:54.759908+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:54.759908+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20363, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.768869+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:54.768869+0000', 'last_peered': '2024-09-16T04:58:54.768869+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:54.768869+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19989, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.760339+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:54.760339+0000', 'last_peered': '2024-09-16T04:58:54.760339+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:54.760339+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:59:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'20532", 'reported_seq': 24619, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:10.021098+0000', 'last_change': '2024-09-16T04:58:53.366597+0000', 'last_active': '2024-09-16T04:59:10.021098+0000', 'last_peered': '2024-09-16T04:59:10.021098+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:59:10.021098+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'10500", 'ondisk_log_start': "47'10500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1851, 'num_objects': 3, 'num_object_clones': 0, 'num_object_copies': 6, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1533, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20529, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1988, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1530}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:14.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 20571, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766610+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:54.766610+0000', 'last_peered': '2024-09-16T04:58:54.766610+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:54.766610+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:14.850 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:15.222 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:15.222 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25437/18 objects degraded (141316.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":703,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1489481728,"bytes_avail":288420810752,"bytes_total":289910292480,"degraded_objects":25437,"degraded_total":18,"degraded_ratio":1413.166666666667,"recovering_objects_per_sec":97,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1170,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:59:15.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:59:15.241 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:15.550 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:15.551 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:15.570 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-16T04:59:14.550657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25437,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15306,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454572,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660948,"statfs":{"total":289910292480,"available":288420810752,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1170,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003977"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758816+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758816+0000","last_peered":"2024-09-16T04:58:54.758816+0000","last_clean":"2024-09-16T04:58:54.758816+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758816+0000","last_undegraded":"2024-09-16T04:58:54.758816+0000","last_fullsized":"2024-09-16T04:58:54.758816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":24619,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:10.021098+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:10.021098+0000","last_peered":"2024-09-16T04:59:10.021098+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:10.021098+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1988,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1530}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":86,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760873+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760873+0000","last_peered":"2024-09-16T04:58:54.760873+0000","last_clean":"2024-09-16T04:58:54.760873+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760873+0000","last_undegraded":"2024-09-16T04:58:54.760873+0000","last_fullsized":"2024-09-16T04:58:54.760873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25437,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15305,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593843,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:18.572 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:18.903 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:18.903 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:18.921 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-16T04:59:18.552473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15783,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454572,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660948,"statfs":{"total":289910292480,"available":288420810752,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-902,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":902,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004679"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758816+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758816+0000","last_peered":"2024-09-16T04:58:54.758816+0000","last_clean":"2024-09-16T04:58:54.758816+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758816+0000","last_undegraded":"2024-09-16T04:58:54.758816+0000","last_fullsized":"2024-09-16T04:58:54.758816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":25573,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:15.020892+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:15.020892+0000","last_peered":"2024-09-16T04:59:15.020892+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:15.020892+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2465,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1053}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760949+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760949+0000","last_peered":"2024-09-16T04:58:54.760949+0000","last_clean":"2024-09-16T04:58:54.760949+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760949+0000","last_undegraded":"2024-09-16T04:58:54.760949+0000","last_fullsized":"2024-09-16T04:58:54.760949+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15782,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593844,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:18.923 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:19.245 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:19.263 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-16T04:59:18.552473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15783,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454572,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660948,"statfs":{"total":289910292480,"available":288420810752,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-902,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":902,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004679"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758816+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758816+0000","last_peered":"2024-09-16T04:58:54.758816+0000","last_clean":"2024-09-16T04:58:54.758816+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758816+0000","last_undegraded":"2024-09-16T04:58:54.758816+0000","last_fullsized":"2024-09-16T04:58:54.758816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":25573,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:15.020892+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:15.020892+0000","last_peered":"2024-09-16T04:59:15.020892+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:15.020892+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2465,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1053}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760949+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760949+0000","last_peered":"2024-09-16T04:58:54.760949+0000","last_clean":"2024-09-16T04:58:54.760949+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760949+0000","last_undegraded":"2024-09-16T04:58:54.760949+0000","last_fullsized":"2024-09-16T04:58:54.760949+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15782,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593844,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:19.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:19.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20012, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.759098+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:54.759098+0000', 'last_peered': '2024-09-16T04:58:54.759098+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:54.759098+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:19.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:19.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20461, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766815+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:54.766815+0000', 'last_peered': '2024-09-16T04:58:54.766815+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:54.766815+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:19.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:19.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20023, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766509+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:54.766509+0000', 'last_peered': '2024-09-16T04:58:54.766509+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:54.766509+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:19.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:19.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20324, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.759908+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:54.759908+0000', 'last_peered': '2024-09-16T04:58:54.759908+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:54.759908+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:19.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:19.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20363, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.768869+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:54.768869+0000', 'last_peered': '2024-09-16T04:58:54.768869+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:54.768869+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:19.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:19.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19989, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.760339+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:54.760339+0000', 'last_peered': '2024-09-16T04:58:54.760339+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:54.760339+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:19.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:59:19.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'20532", 'reported_seq': 25573, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:15.020892+0000', 'last_change': '2024-09-16T04:58:53.366597+0000', 'last_active': '2024-09-16T04:59:15.020892+0000', 'last_peered': '2024-09-16T04:59:15.020892+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:59:15.020892+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'10500", 'ondisk_log_start': "47'10500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1851, 'num_objects': 3, 'num_object_clones': 0, 'num_object_copies': 6, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1056, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20529, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2465, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1053}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:19.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:19.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 20571, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766610+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:54.766610+0000', 'last_peered': '2024-09-16T04:58:54.766610+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:54.766610+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:19.266 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:19.636 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24960/18 objects degraded (138666.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":707,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1489481728,"bytes_avail":288420810752,"bytes_total":289910292480,"degraded_objects":24960,"degraded_total":18,"degraded_ratio":1386.666666666667,"recovering_objects_per_sec":75,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":902,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:59:19.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:59:19.655 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:19.966 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:19.967 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:19.985 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-16T04:59:18.552473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15783,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454572,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660948,"statfs":{"total":289910292480,"available":288420810752,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-902,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":902,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004679"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758816+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758816+0000","last_peered":"2024-09-16T04:58:54.758816+0000","last_clean":"2024-09-16T04:58:54.758816+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758816+0000","last_undegraded":"2024-09-16T04:58:54.758816+0000","last_fullsized":"2024-09-16T04:58:54.758816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":25573,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:15.020892+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:15.020892+0000","last_peered":"2024-09-16T04:59:15.020892+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:15.020892+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2465,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1053}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760949+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760949+0000","last_peered":"2024-09-16T04:58:54.760949+0000","last_clean":"2024-09-16T04:58:54.760949+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760949+0000","last_undegraded":"2024-09-16T04:58:54.760949+0000","last_fullsized":"2024-09-16T04:58:54.760949+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15782,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593844,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473660,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898180,"statfs":{"total":96636764160,"available":96151736320,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:22.987 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:23.299 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:23.299 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:23.318 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-09-16T04:59:22.553983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16340,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446252,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281669268,"statfs":{"total":289910292480,"available":288429330432,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1034,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004754"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758816+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758816+0000","last_peered":"2024-09-16T04:58:54.758816+0000","last_clean":"2024-09-16T04:58:54.758816+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758816+0000","last_undegraded":"2024-09-16T04:58:54.758816+0000","last_fullsized":"2024-09-16T04:58:54.758816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":26687,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:20.022464+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:20.022464+0000","last_peered":"2024-09-16T04:59:20.022464+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:20.022464+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3022,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-496}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760949+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760949+0000","last_peered":"2024-09-16T04:58:54.760949+0000","last_clean":"2024-09-16T04:58:54.760949+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760949+0000","last_undegraded":"2024-09-16T04:58:54.760949+0000","last_fullsized":"2024-09-16T04:58:54.760949+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16339,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593845,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465340,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906500,"statfs":{"total":96636764160,"available":96160256000,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:23.320 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:23.684 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:23.703 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-09-16T04:59:22.553983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16340,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446252,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281669268,"statfs":{"total":289910292480,"available":288429330432,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1034,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004754"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758816+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758816+0000","last_peered":"2024-09-16T04:58:54.758816+0000","last_clean":"2024-09-16T04:58:54.758816+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758816+0000","last_undegraded":"2024-09-16T04:58:54.758816+0000","last_fullsized":"2024-09-16T04:58:54.758816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":26687,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:20.022464+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:20.022464+0000","last_peered":"2024-09-16T04:59:20.022464+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:20.022464+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3022,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-496}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760949+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760949+0000","last_peered":"2024-09-16T04:58:54.760949+0000","last_clean":"2024-09-16T04:58:54.760949+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760949+0000","last_undegraded":"2024-09-16T04:58:54.760949+0000","last_fullsized":"2024-09-16T04:58:54.760949+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16339,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593845,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465340,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906500,"statfs":{"total":96636764160,"available":96160256000,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:23.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:23.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20012, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.759098+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:58:54.759098+0000', 'last_peered': '2024-09-16T04:58:54.759098+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:58:54.759098+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:23.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:23.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20461, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766815+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:58:54.766815+0000', 'last_peered': '2024-09-16T04:58:54.766815+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:58:54.766815+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:23.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:23.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20023, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766509+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:58:54.766509+0000', 'last_peered': '2024-09-16T04:58:54.766509+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:58:54.766509+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:23.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:23.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20324, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.759908+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:58:54.759908+0000', 'last_peered': '2024-09-16T04:58:54.759908+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:58:54.759908+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:23.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:23.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20363, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.768869+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:58:54.768869+0000', 'last_peered': '2024-09-16T04:58:54.768869+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:58:54.768869+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:23.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:23.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19989, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.760339+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:58:54.760339+0000', 'last_peered': '2024-09-16T04:58:54.760339+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:58:54.760339+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:23.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:59:23.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'20532", 'reported_seq': 26687, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:20.022464+0000', 'last_change': '2024-09-16T04:58:53.366597+0000', 'last_active': '2024-09-16T04:59:20.022464+0000', 'last_peered': '2024-09-16T04:59:20.022464+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:59:20.022464+0000', 'last_undegraded': '2024-09-16T04:49:37.771433+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 44, 'log_start': "47'10500", 'ondisk_log_start': "47'10500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1851, 'num_objects': 3, 'num_object_clones': 0, 'num_object_copies': 6, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20529, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3022, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -496}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:23.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:23.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 20571, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:58:54.766610+0000', 'last_change': '2024-09-16T04:49:38.239062+0000', 'last_active': '2024-09-16T04:58:54.766610+0000', 'last_peered': '2024-09-16T04:58:54.766610+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:58:54.766610+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3527, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:23.706 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:24.071 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:24.072 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24403/18 objects degraded (135572.222%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":711,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1480962048,"bytes_avail":288429330432,"bytes_total":289910292480,"degraded_objects":24403,"degraded_total":18,"degraded_ratio":1355.7222222222222,"recovering_objects_per_sec":86,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1034,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:59:24.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:59:24.091 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:24.400 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:24.401 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:24.420 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-09-16T04:59:22.553983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16340,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446252,"kb_used_data":164204,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281669268,"statfs":{"total":289910292480,"available":288429330432,"internally_reserved":0,"allocated":168144896,"data_stored":167199831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1034,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1034,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004754"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26521,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.924942+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:58:54.924942+0000","last_peered":"2024-09-16T04:58:54.924942+0000","last_clean":"2024-09-16T04:58:54.924942+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:58:54.924942+0000","last_undegraded":"2024-09-16T04:58:54.924942+0000","last_fullsized":"2024-09-16T04:58:54.924942+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20125,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.765755+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:58:54.765755+0000","last_peered":"2024-09-16T04:58:54.765755+0000","last_clean":"2024-09-16T04:58:54.765755+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:58:54.765755+0000","last_undegraded":"2024-09-16T04:58:54.765755+0000","last_fullsized":"2024-09-16T04:58:54.765755+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20012,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759098+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:58:54.759098+0000","last_peered":"2024-09-16T04:58:54.759098+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:58:54.759098+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20429,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.759103+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:58:54.759103+0000","last_peered":"2024-09-16T04:58:54.759103+0000","last_clean":"2024-09-16T04:58:54.759103+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:58:54.759103+0000","last_undegraded":"2024-09-16T04:58:54.759103+0000","last_fullsized":"2024-09-16T04:58:54.759103+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766936+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:58:54.766936+0000","last_peered":"2024-09-16T04:58:54.766936+0000","last_clean":"2024-09-16T04:58:54.766936+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:58:54.766936+0000","last_undegraded":"2024-09-16T04:58:54.766936+0000","last_fullsized":"2024-09-16T04:58:54.766936+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20461,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766815+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:58:54.766815+0000","last_peered":"2024-09-16T04:58:54.766815+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:58:54.766815+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20023,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766509+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:58:54.766509+0000","last_peered":"2024-09-16T04:58:54.766509+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:58:54.766509+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766404+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:58:54.766404+0000","last_peered":"2024-09-16T04:58:54.766404+0000","last_clean":"2024-09-16T04:58:54.766404+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:58:54.766404+0000","last_undegraded":"2024-09-16T04:58:54.766404+0000","last_fullsized":"2024-09-16T04:58:54.766404+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.762059+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:58:54.762059+0000","last_peered":"2024-09-16T04:58:54.762059+0000","last_clean":"2024-09-16T04:58:54.762059+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:58:54.762059+0000","last_undegraded":"2024-09-16T04:58:54.762059+0000","last_fullsized":"2024-09-16T04:58:54.762059+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20324,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.759908+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:58:54.759908+0000","last_peered":"2024-09-16T04:58:54.759908+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:58:54.759908+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758816+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:58:54.758816+0000","last_peered":"2024-09-16T04:58:54.758816+0000","last_clean":"2024-09-16T04:58:54.758816+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:58:54.758816+0000","last_undegraded":"2024-09-16T04:58:54.758816+0000","last_fullsized":"2024-09-16T04:58:54.758816+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20363,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.768869+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:58:54.768869+0000","last_peered":"2024-09-16T04:58:54.768869+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:58:54.768869+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.766631+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:58:54.766631+0000","last_peered":"2024-09-16T04:58:54.766631+0000","last_clean":"2024-09-16T04:58:54.766631+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:58:54.766631+0000","last_undegraded":"2024-09-16T04:58:54.766631+0000","last_fullsized":"2024-09-16T04:58:54.766631+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26839,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758790+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:58:54.758790+0000","last_peered":"2024-09-16T04:58:54.758790+0000","last_clean":"2024-09-16T04:58:54.758790+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:58:54.758790+0000","last_undegraded":"2024-09-16T04:58:54.758790+0000","last_fullsized":"2024-09-16T04:58:54.758790+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:58:57.679062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763083+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:58:54.763083+0000","last_peered":"2024-09-16T04:58:54.763083+0000","last_clean":"2024-09-16T04:58:54.763083+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:58:54.763083+0000","last_undegraded":"2024-09-16T04:58:54.763083+0000","last_fullsized":"2024-09-16T04:58:54.763083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26850,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.758547+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:58:54.758547+0000","last_peered":"2024-09-16T04:58:54.758547+0000","last_clean":"2024-09-16T04:58:54.758547+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:58:54.758547+0000","last_undegraded":"2024-09-16T04:58:54.758547+0000","last_fullsized":"2024-09-16T04:58:54.758547+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19989,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.760339+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:58:54.760339+0000","last_peered":"2024-09-16T04:58:54.760339+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:58:54.760339+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763363+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:58:54.763363+0000","last_peered":"2024-09-16T04:58:54.763363+0000","last_clean":"2024-09-16T04:58:54.763363+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:58:54.763363+0000","last_undegraded":"2024-09-16T04:58:54.763363+0000","last_fullsized":"2024-09-16T04:58:54.763363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":26687,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:20.022464+0000","last_change":"2024-09-16T04:58:53.366597+0000","last_active":"2024-09-16T04:59:20.022464+0000","last_peered":"2024-09-16T04:59:20.022464+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:20.022464+0000","last_undegraded":"2024-09-16T04:49:37.771433+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":44,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3022,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-496}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.760949+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:58:54.760949+0000","last_peered":"2024-09-16T04:58:54.760949+0000","last_clean":"2024-09-16T04:58:54.760949+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:58:54.760949+0000","last_undegraded":"2024-09-16T04:58:54.760949+0000","last_fullsized":"2024-09-16T04:58:54.760949+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27007,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763367+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:58:54.763367+0000","last_peered":"2024-09-16T04:58:54.763367+0000","last_clean":"2024-09-16T04:58:54.763367+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:58:54.763367+0000","last_undegraded":"2024-09-16T04:58:54.763367+0000","last_fullsized":"2024-09-16T04:58:54.763367+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763453+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:58:54.763453+0000","last_peered":"2024-09-16T04:58:54.763453+0000","last_clean":"2024-09-16T04:58:54.763453+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:58:54.763453+0000","last_undegraded":"2024-09-16T04:58:54.763453+0000","last_fullsized":"2024-09-16T04:58:54.763453+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T04:58:54.763562+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:58:54.763562+0000","last_peered":"2024-09-16T04:58:54.763562+0000","last_clean":"2024-09-16T04:58:54.763562+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:58:54.763562+0000","last_undegraded":"2024-09-16T04:58:54.763562+0000","last_fullsized":"2024-09-16T04:58:54.763562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20571,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:58:54.766610+0000","last_change":"2024-09-16T04:49:38.239062+0000","last_active":"2024-09-16T04:58:54.766610+0000","last_peered":"2024-09-16T04:58:54.766610+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:58:54.766610+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16339,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166805504,"data_stored":166749840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493180,"kb_used_data":82468,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878660,"statfs":{"total":96636764160,"available":96131747840,"internally_reserved":0,"allocated":84447232,"data_stored":84147286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593845,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465340,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906500,"statfs":{"total":96636764160,"available":96160256000,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487732,"kb_used_data":81244,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884108,"statfs":{"total":96636764160,"available":96137326592,"internally_reserved":0,"allocated":83193856,"data_stored":82893910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:27.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:27.772 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:27.772 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:27.791 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-09-16T04:59:26.555358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23827,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16916,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450420,"kb_used_data":164220,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665100,"statfs":{"total":289910292480,"available":288425062400,"internally_reserved":0,"allocated":168161280,"data_stored":167205815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002885"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26523,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.721661+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:24.721661+0000","last_peered":"2024-09-16T04:59:24.721661+0000","last_clean":"2024-09-16T04:59:24.721661+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:24.721661+0000","last_undegraded":"2024-09-16T04:59:24.721661+0000","last_fullsized":"2024-09-16T04:59:24.721661+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20127,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.729747+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:24.729747+0000","last_peered":"2024-09-16T04:59:24.729747+0000","last_clean":"2024-09-16T04:59:24.729747+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:24.729747+0000","last_undegraded":"2024-09-16T04:59:24.729747+0000","last_fullsized":"2024-09-16T04:59:24.729747+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20014,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.717528+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:24.717528+0000","last_peered":"2024-09-16T04:59:24.717528+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:24.717528+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20431,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.717350+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:24.717350+0000","last_peered":"2024-09-16T04:59:24.717350+0000","last_clean":"2024-09-16T04:59:24.717350+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:24.717350+0000","last_undegraded":"2024-09-16T04:59:24.717350+0000","last_fullsized":"2024-09-16T04:59:24.717350+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20380,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.720967+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:24.720967+0000","last_peered":"2024-09-16T04:59:24.720967+0000","last_clean":"2024-09-16T04:59:24.720967+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:24.720967+0000","last_undegraded":"2024-09-16T04:59:24.720967+0000","last_fullsized":"2024-09-16T04:59:24.720967+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20464,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.720428+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:24.720428+0000","last_peered":"2024-09-16T04:59:24.720428+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:24.720428+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20026,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.720665+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:24.720665+0000","last_peered":"2024-09-16T04:59:24.720665+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:24.720665+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20249,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.720264+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:24.720264+0000","last_peered":"2024-09-16T04:59:24.720264+0000","last_clean":"2024-09-16T04:59:24.720264+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:24.720264+0000","last_undegraded":"2024-09-16T04:59:24.720264+0000","last_fullsized":"2024-09-16T04:59:24.720264+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.712613+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:24.712613+0000","last_peered":"2024-09-16T04:59:24.712613+0000","last_clean":"2024-09-16T04:59:24.712613+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:24.712613+0000","last_undegraded":"2024-09-16T04:59:24.712613+0000","last_fullsized":"2024-09-16T04:59:24.712613+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20326,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.717831+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:24.717831+0000","last_peered":"2024-09-16T04:59:24.717831+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:24.717831+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.716236+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:24.716236+0000","last_peered":"2024-09-16T04:59:24.716236+0000","last_clean":"2024-09-16T04:59:24.716236+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:24.716236+0000","last_undegraded":"2024-09-16T04:59:24.716236+0000","last_fullsized":"2024-09-16T04:59:24.716236+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20365,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.721376+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:24.721376+0000","last_peered":"2024-09-16T04:59:24.721376+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:24.721376+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.718430+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:24.718430+0000","last_peered":"2024-09-16T04:59:24.718430+0000","last_clean":"2024-09-16T04:59:24.718430+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:24.718430+0000","last_undegraded":"2024-09-16T04:59:24.718430+0000","last_fullsized":"2024-09-16T04:59:24.718430+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26841,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.716787+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:24.716787+0000","last_peered":"2024-09-16T04:59:24.716787+0000","last_clean":"2024-09-16T04:59:24.716787+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:24.716787+0000","last_undegraded":"2024-09-16T04:59:24.716787+0000","last_fullsized":"2024-09-16T04:59:24.716787+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:24.896079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.712787+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:24.712787+0000","last_peered":"2024-09-16T04:59:24.712787+0000","last_clean":"2024-09-16T04:59:24.712787+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:24.712787+0000","last_undegraded":"2024-09-16T04:59:24.712787+0000","last_fullsized":"2024-09-16T04:59:24.712787+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26852,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.716284+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:24.716284+0000","last_peered":"2024-09-16T04:59:24.716284+0000","last_clean":"2024-09-16T04:59:24.716284+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:24.716284+0000","last_undegraded":"2024-09-16T04:59:24.716284+0000","last_fullsized":"2024-09-16T04:59:24.716284+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19991,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.717635+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:24.717635+0000","last_peered":"2024-09-16T04:59:24.717635+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:24.717635+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.713806+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:24.713806+0000","last_peered":"2024-09-16T04:59:24.713806+0000","last_clean":"2024-09-16T04:59:24.713806+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:24.713806+0000","last_undegraded":"2024-09-16T04:59:24.713806+0000","last_fullsized":"2024-09-16T04:59:24.713806+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20650,"reported_epoch":54,"state":"peering","last_fresh":"2024-09-16T04:59:24.056766+0000","last_change":"2024-09-16T04:59:24.925922+0000","last_active":"2024-09-16T04:59:24.925922+0000","last_peered":"2024-09-16T04:59:24.056766+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:24.056766+0000","last_undegraded":"2024-09-16T04:59:24.056766+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.718409+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:24.718409+0000","last_peered":"2024-09-16T04:59:24.718409+0000","last_clean":"2024-09-16T04:59:24.718409+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:24.718409+0000","last_undegraded":"2024-09-16T04:59:24.718409+0000","last_fullsized":"2024-09-16T04:59:24.718409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27009,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.713786+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:24.713786+0000","last_peered":"2024-09-16T04:59:24.713786+0000","last_clean":"2024-09-16T04:59:24.713786+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:24.713786+0000","last_undegraded":"2024-09-16T04:59:24.713786+0000","last_fullsized":"2024-09-16T04:59:24.713786+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.713650+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:24.713650+0000","last_peered":"2024-09-16T04:59:24.713650+0000","last_clean":"2024-09-16T04:59:24.713650+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:24.713650+0000","last_undegraded":"2024-09-16T04:59:24.713650+0000","last_fullsized":"2024-09-16T04:59:24.713650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.713997+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:24.713997+0000","last_peered":"2024-09-16T04:59:24.713997+0000","last_clean":"2024-09-16T04:59:24.713997+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:24.713997+0000","last_undegraded":"2024-09-16T04:59:24.713997+0000","last_fullsized":"2024-09-16T04:59:24.713997+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20728,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.099132+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:25.099132+0000","last_peered":"2024-09-16T04:59:25.099132+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:25.099132+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3450,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3449}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23827,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16915,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493184,"kb_used_data":82476,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878656,"statfs":{"total":96636764160,"available":96131743744,"internally_reserved":0,"allocated":84455424,"data_stored":84150278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593846,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469500,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902340,"statfs":{"total":96636764160,"available":96155996160,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487736,"kb_used_data":81252,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884104,"statfs":{"total":96636764160,"available":96137322496,"internally_reserved":0,"allocated":83202048,"data_stored":82896902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:27.793 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:28.139 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:28.140 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:28.159 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-09-16T04:59:26.555358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23827,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16916,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450420,"kb_used_data":164220,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665100,"statfs":{"total":289910292480,"available":288425062400,"internally_reserved":0,"allocated":168161280,"data_stored":167205815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002885"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26523,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.721661+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:24.721661+0000","last_peered":"2024-09-16T04:59:24.721661+0000","last_clean":"2024-09-16T04:59:24.721661+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:24.721661+0000","last_undegraded":"2024-09-16T04:59:24.721661+0000","last_fullsized":"2024-09-16T04:59:24.721661+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20127,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.729747+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:24.729747+0000","last_peered":"2024-09-16T04:59:24.729747+0000","last_clean":"2024-09-16T04:59:24.729747+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:24.729747+0000","last_undegraded":"2024-09-16T04:59:24.729747+0000","last_fullsized":"2024-09-16T04:59:24.729747+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20014,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.717528+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:24.717528+0000","last_peered":"2024-09-16T04:59:24.717528+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:24.717528+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20431,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.717350+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:24.717350+0000","last_peered":"2024-09-16T04:59:24.717350+0000","last_clean":"2024-09-16T04:59:24.717350+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:24.717350+0000","last_undegraded":"2024-09-16T04:59:24.717350+0000","last_fullsized":"2024-09-16T04:59:24.717350+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20380,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.720967+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:24.720967+0000","last_peered":"2024-09-16T04:59:24.720967+0000","last_clean":"2024-09-16T04:59:24.720967+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:24.720967+0000","last_undegraded":"2024-09-16T04:59:24.720967+0000","last_fullsized":"2024-09-16T04:59:24.720967+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20464,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.720428+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:24.720428+0000","last_peered":"2024-09-16T04:59:24.720428+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:24.720428+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20026,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.720665+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:24.720665+0000","last_peered":"2024-09-16T04:59:24.720665+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:24.720665+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20249,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.720264+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:24.720264+0000","last_peered":"2024-09-16T04:59:24.720264+0000","last_clean":"2024-09-16T04:59:24.720264+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:24.720264+0000","last_undegraded":"2024-09-16T04:59:24.720264+0000","last_fullsized":"2024-09-16T04:59:24.720264+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.712613+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:24.712613+0000","last_peered":"2024-09-16T04:59:24.712613+0000","last_clean":"2024-09-16T04:59:24.712613+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:24.712613+0000","last_undegraded":"2024-09-16T04:59:24.712613+0000","last_fullsized":"2024-09-16T04:59:24.712613+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20326,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.717831+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:24.717831+0000","last_peered":"2024-09-16T04:59:24.717831+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:24.717831+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.716236+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:24.716236+0000","last_peered":"2024-09-16T04:59:24.716236+0000","last_clean":"2024-09-16T04:59:24.716236+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:24.716236+0000","last_undegraded":"2024-09-16T04:59:24.716236+0000","last_fullsized":"2024-09-16T04:59:24.716236+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20365,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.721376+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:24.721376+0000","last_peered":"2024-09-16T04:59:24.721376+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:24.721376+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.718430+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:24.718430+0000","last_peered":"2024-09-16T04:59:24.718430+0000","last_clean":"2024-09-16T04:59:24.718430+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:24.718430+0000","last_undegraded":"2024-09-16T04:59:24.718430+0000","last_fullsized":"2024-09-16T04:59:24.718430+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26841,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.716787+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:24.716787+0000","last_peered":"2024-09-16T04:59:24.716787+0000","last_clean":"2024-09-16T04:59:24.716787+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:24.716787+0000","last_undegraded":"2024-09-16T04:59:24.716787+0000","last_fullsized":"2024-09-16T04:59:24.716787+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:24.896079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.712787+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:24.712787+0000","last_peered":"2024-09-16T04:59:24.712787+0000","last_clean":"2024-09-16T04:59:24.712787+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:24.712787+0000","last_undegraded":"2024-09-16T04:59:24.712787+0000","last_fullsized":"2024-09-16T04:59:24.712787+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26852,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.716284+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:24.716284+0000","last_peered":"2024-09-16T04:59:24.716284+0000","last_clean":"2024-09-16T04:59:24.716284+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:24.716284+0000","last_undegraded":"2024-09-16T04:59:24.716284+0000","last_fullsized":"2024-09-16T04:59:24.716284+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19991,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.717635+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:24.717635+0000","last_peered":"2024-09-16T04:59:24.717635+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:24.717635+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.713806+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:24.713806+0000","last_peered":"2024-09-16T04:59:24.713806+0000","last_clean":"2024-09-16T04:59:24.713806+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:24.713806+0000","last_undegraded":"2024-09-16T04:59:24.713806+0000","last_fullsized":"2024-09-16T04:59:24.713806+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20650,"reported_epoch":54,"state":"peering","last_fresh":"2024-09-16T04:59:24.056766+0000","last_change":"2024-09-16T04:59:24.925922+0000","last_active":"2024-09-16T04:59:24.925922+0000","last_peered":"2024-09-16T04:59:24.056766+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:24.056766+0000","last_undegraded":"2024-09-16T04:59:24.056766+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.718409+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:24.718409+0000","last_peered":"2024-09-16T04:59:24.718409+0000","last_clean":"2024-09-16T04:59:24.718409+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:24.718409+0000","last_undegraded":"2024-09-16T04:59:24.718409+0000","last_fullsized":"2024-09-16T04:59:24.718409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27009,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.713786+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:24.713786+0000","last_peered":"2024-09-16T04:59:24.713786+0000","last_clean":"2024-09-16T04:59:24.713786+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:24.713786+0000","last_undegraded":"2024-09-16T04:59:24.713786+0000","last_fullsized":"2024-09-16T04:59:24.713786+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.713650+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:24.713650+0000","last_peered":"2024-09-16T04:59:24.713650+0000","last_clean":"2024-09-16T04:59:24.713650+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:24.713650+0000","last_undegraded":"2024-09-16T04:59:24.713650+0000","last_fullsized":"2024-09-16T04:59:24.713650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.713997+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:24.713997+0000","last_peered":"2024-09-16T04:59:24.713997+0000","last_clean":"2024-09-16T04:59:24.713997+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:24.713997+0000","last_undegraded":"2024-09-16T04:59:24.713997+0000","last_fullsized":"2024-09-16T04:59:24.713997+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20728,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.099132+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:25.099132+0000","last_peered":"2024-09-16T04:59:25.099132+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:25.099132+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3450,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3449}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23827,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16915,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493184,"kb_used_data":82476,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878656,"statfs":{"total":96636764160,"available":96131743744,"internally_reserved":0,"allocated":84455424,"data_stored":84150278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593846,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469500,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902340,"statfs":{"total":96636764160,"available":96155996160,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487736,"kb_used_data":81252,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884104,"statfs":{"total":96636764160,"available":96137322496,"internally_reserved":0,"allocated":83202048,"data_stored":82896902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:28.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:28.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20014, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:24.717528+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:24.717528+0000', 'last_peered': '2024-09-16T04:59:24.717528+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:24.717528+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:28.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:28.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20464, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:24.720428+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:24.720428+0000', 'last_peered': '2024-09-16T04:59:24.720428+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:24.720428+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:28.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:28.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20026, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:24.720665+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:24.720665+0000', 'last_peered': '2024-09-16T04:59:24.720665+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:24.720665+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20326, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:24.717831+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:24.717831+0000', 'last_peered': '2024-09-16T04:59:24.717831+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:24.717831+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20365, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:24.721376+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:24.721376+0000', 'last_peered': '2024-09-16T04:59:24.721376+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:24.721376+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19991, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:24.717635+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:59:24.717635+0000', 'last_peered': '2024-09-16T04:59:24.717635+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:59:24.717635+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-16T04:59:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "51'20532", 'reported_seq': 20650, 'reported_epoch': 54, 'state': 'peering', 'last_fresh': '2024-09-16T04:59:24.056766+0000', 'last_change': '2024-09-16T04:59:24.925922+0000', 'last_active': '2024-09-16T04:59:24.925922+0000', 'last_peered': '2024-09-16T04:59:24.056766+0000', 'last_clean': '2024-09-16T04:49:35.584799+0000', 'last_became_active': '2024-09-16T04:49:38.162709+0000', 'last_became_peered': '2024-09-16T04:49:38.162709+0000', 'last_unstale': '2024-09-16T04:59:24.056766+0000', 'last_undegraded': '2024-09-16T04:59:24.056766+0000', 'last_fullsized': '2024-09-16T04:49:37.645131+0000', 'mapping_epoch': 54, 'log_start': "47'10500", 'ondisk_log_start': "47'10500", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 399, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1851, 'num_objects': 3, 'num_object_clones': 0, 'num_object_copies': 6, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20529, 'num_write_kb': 41052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3521, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T04:59:28.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:28.162 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 20728, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.099132+0000', 'last_change': '2024-09-16T04:59:24.057372+0000', 'last_active': '2024-09-16T04:59:25.099132+0000', 'last_peered': '2024-09-16T04:59:25.099132+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:59:25.099132+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3450, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3449}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:28.162 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:28.576 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23827/18 objects degraded (132372.222%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":716,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"peering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1485230080,"bytes_avail":288425062400,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":23827,"degraded_total":18,"degraded_ratio":1323.7222222222222,"recovering_objects_per_sec":141,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1133,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:59:28.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:59:28.601 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:28.951 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:28.975 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-09-16T04:59:28.555983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23827,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16916,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450420,"kb_used_data":164220,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665100,"statfs":{"total":289910292480,"available":288425062400,"internally_reserved":0,"allocated":168161280,"data_stored":167205815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-576,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":576,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002770"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26523,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.721661+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:24.721661+0000","last_peered":"2024-09-16T04:59:24.721661+0000","last_clean":"2024-09-16T04:59:24.721661+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:24.721661+0000","last_undegraded":"2024-09-16T04:59:24.721661+0000","last_fullsized":"2024-09-16T04:59:24.721661+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20127,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.729747+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:24.729747+0000","last_peered":"2024-09-16T04:59:24.729747+0000","last_clean":"2024-09-16T04:59:24.729747+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:24.729747+0000","last_undegraded":"2024-09-16T04:59:24.729747+0000","last_fullsized":"2024-09-16T04:59:24.729747+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20014,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.717528+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:24.717528+0000","last_peered":"2024-09-16T04:59:24.717528+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:24.717528+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20431,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.717350+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:24.717350+0000","last_peered":"2024-09-16T04:59:24.717350+0000","last_clean":"2024-09-16T04:59:24.717350+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:24.717350+0000","last_undegraded":"2024-09-16T04:59:24.717350+0000","last_fullsized":"2024-09-16T04:59:24.717350+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20380,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.720967+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:24.720967+0000","last_peered":"2024-09-16T04:59:24.720967+0000","last_clean":"2024-09-16T04:59:24.720967+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:24.720967+0000","last_undegraded":"2024-09-16T04:59:24.720967+0000","last_fullsized":"2024-09-16T04:59:24.720967+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20464,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.720428+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:24.720428+0000","last_peered":"2024-09-16T04:59:24.720428+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:24.720428+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20026,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.720665+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:24.720665+0000","last_peered":"2024-09-16T04:59:24.720665+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:24.720665+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20249,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.720264+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:24.720264+0000","last_peered":"2024-09-16T04:59:24.720264+0000","last_clean":"2024-09-16T04:59:24.720264+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:24.720264+0000","last_undegraded":"2024-09-16T04:59:24.720264+0000","last_fullsized":"2024-09-16T04:59:24.720264+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.712613+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:24.712613+0000","last_peered":"2024-09-16T04:59:24.712613+0000","last_clean":"2024-09-16T04:59:24.712613+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:24.712613+0000","last_undegraded":"2024-09-16T04:59:24.712613+0000","last_fullsized":"2024-09-16T04:59:24.712613+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20326,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.717831+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:24.717831+0000","last_peered":"2024-09-16T04:59:24.717831+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:24.717831+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.716236+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:24.716236+0000","last_peered":"2024-09-16T04:59:24.716236+0000","last_clean":"2024-09-16T04:59:24.716236+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:24.716236+0000","last_undegraded":"2024-09-16T04:59:24.716236+0000","last_fullsized":"2024-09-16T04:59:24.716236+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20365,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.721376+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:24.721376+0000","last_peered":"2024-09-16T04:59:24.721376+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:24.721376+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.718430+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:24.718430+0000","last_peered":"2024-09-16T04:59:24.718430+0000","last_clean":"2024-09-16T04:59:24.718430+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:24.718430+0000","last_undegraded":"2024-09-16T04:59:24.718430+0000","last_fullsized":"2024-09-16T04:59:24.718430+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26841,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.716787+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:24.716787+0000","last_peered":"2024-09-16T04:59:24.716787+0000","last_clean":"2024-09-16T04:59:24.716787+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:24.716787+0000","last_undegraded":"2024-09-16T04:59:24.716787+0000","last_fullsized":"2024-09-16T04:59:24.716787+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:24.896079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.712787+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:24.712787+0000","last_peered":"2024-09-16T04:59:24.712787+0000","last_clean":"2024-09-16T04:59:24.712787+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:24.712787+0000","last_undegraded":"2024-09-16T04:59:24.712787+0000","last_fullsized":"2024-09-16T04:59:24.712787+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26852,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.716284+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:24.716284+0000","last_peered":"2024-09-16T04:59:24.716284+0000","last_clean":"2024-09-16T04:59:24.716284+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:24.716284+0000","last_undegraded":"2024-09-16T04:59:24.716284+0000","last_fullsized":"2024-09-16T04:59:24.716284+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19991,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:24.717635+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:24.717635+0000","last_peered":"2024-09-16T04:59:24.717635+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:24.717635+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.713806+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:24.713806+0000","last_peered":"2024-09-16T04:59:24.713806+0000","last_clean":"2024-09-16T04:59:24.713806+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:24.713806+0000","last_undegraded":"2024-09-16T04:59:24.713806+0000","last_fullsized":"2024-09-16T04:59:24.713806+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20650,"reported_epoch":54,"state":"peering","last_fresh":"2024-09-16T04:59:24.056766+0000","last_change":"2024-09-16T04:59:24.925922+0000","last_active":"2024-09-16T04:59:24.925922+0000","last_peered":"2024-09-16T04:59:24.056766+0000","last_clean":"2024-09-16T04:49:35.584799+0000","last_became_active":"2024-09-16T04:49:38.162709+0000","last_became_peered":"2024-09-16T04:49:38.162709+0000","last_unstale":"2024-09-16T04:59:24.056766+0000","last_undegraded":"2024-09-16T04:59:24.056766+0000","last_fullsized":"2024-09-16T04:49:37.645131+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.718409+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:24.718409+0000","last_peered":"2024-09-16T04:59:24.718409+0000","last_clean":"2024-09-16T04:59:24.718409+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:24.718409+0000","last_undegraded":"2024-09-16T04:59:24.718409+0000","last_fullsized":"2024-09-16T04:59:24.718409+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27009,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.713786+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:24.713786+0000","last_peered":"2024-09-16T04:59:24.713786+0000","last_clean":"2024-09-16T04:59:24.713786+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:24.713786+0000","last_undegraded":"2024-09-16T04:59:24.713786+0000","last_fullsized":"2024-09-16T04:59:24.713786+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.713650+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:24.713650+0000","last_peered":"2024-09-16T04:59:24.713650+0000","last_clean":"2024-09-16T04:59:24.713650+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:24.713650+0000","last_undegraded":"2024-09-16T04:59:24.713650+0000","last_fullsized":"2024-09-16T04:59:24.713650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T04:59:24.713997+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:24.713997+0000","last_peered":"2024-09-16T04:59:24.713997+0000","last_clean":"2024-09-16T04:59:24.713997+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:24.713997+0000","last_undegraded":"2024-09-16T04:59:24.713997+0000","last_fullsized":"2024-09-16T04:59:24.713997+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20728,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.099132+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:25.099132+0000","last_peered":"2024-09-16T04:59:25.099132+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:25.099132+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3450,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3449}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23827,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16915,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493184,"kb_used_data":82476,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878656,"statfs":{"total":96636764160,"available":96131743744,"internally_reserved":0,"allocated":84455424,"data_stored":84150278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593846,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469500,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902340,"statfs":{"total":96636764160,"available":96155996160,"internally_reserved":0,"allocated":503808,"data_stored":158635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487736,"kb_used_data":81252,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884104,"statfs":{"total":96636764160,"available":96137322496,"internally_reserved":0,"allocated":83202048,"data_stored":82896902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:31.978 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:32.318 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:32.319 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:32.341 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-16T04:59:30.556374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17556,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454608,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660912,"statfs":{"total":289910292480,"available":288420773888,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002390"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26525,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728055+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728055+0000","last_peered":"2024-09-16T04:59:25.728055+0000","last_clean":"2024-09-16T04:59:25.728055+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728055+0000","last_undegraded":"2024-09-16T04:59:25.728055+0000","last_fullsized":"2024-09-16T04:59:25.728055+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722484+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722484+0000","last_peered":"2024-09-16T04:59:25.722484+0000","last_clean":"2024-09-16T04:59:25.722484+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722484+0000","last_undegraded":"2024-09-16T04:59:25.722484+0000","last_fullsized":"2024-09-16T04:59:25.722484+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26843,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723968+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.723968+0000","last_peered":"2024-09-16T04:59:25.723968+0000","last_clean":"2024-09-16T04:59:25.723968+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.723968+0000","last_undegraded":"2024-09-16T04:59:25.723968+0000","last_fullsized":"2024-09-16T04:59:25.723968+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":22010,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:30.110711+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:30.110711+0000","last_peered":"2024-09-16T04:59:30.110711+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:30.110711+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2810,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17555,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593847,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473664,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898176,"statfs":{"total":96636764160,"available":96151732224,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:32.343 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:32.712 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:32.732 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-16T04:59:32.557098+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17556,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454608,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660912,"statfs":{"total":289910292480,"available":288420773888,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002379"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26525,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728055+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728055+0000","last_peered":"2024-09-16T04:59:25.728055+0000","last_clean":"2024-09-16T04:59:25.728055+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728055+0000","last_undegraded":"2024-09-16T04:59:25.728055+0000","last_fullsized":"2024-09-16T04:59:25.728055+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722484+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722484+0000","last_peered":"2024-09-16T04:59:25.722484+0000","last_clean":"2024-09-16T04:59:25.722484+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722484+0000","last_undegraded":"2024-09-16T04:59:25.722484+0000","last_fullsized":"2024-09-16T04:59:25.722484+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26843,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723968+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.723968+0000","last_peered":"2024-09-16T04:59:25.723968+0000","last_clean":"2024-09-16T04:59:25.723968+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.723968+0000","last_undegraded":"2024-09-16T04:59:25.723968+0000","last_fullsized":"2024-09-16T04:59:25.723968+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":22010,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:30.110711+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:30.110711+0000","last_peered":"2024-09-16T04:59:30.110711+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:30.110711+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2810,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17555,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593847,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473664,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898176,"statfs":{"total":96636764160,"available":96151732224,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:32.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:32.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20016, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727514+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:25.727514+0000', 'last_peered': '2024-09-16T04:59:25.727514+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:25.727514+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:32.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:32.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20466, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.731663+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:25.731663+0000', 'last_peered': '2024-09-16T04:59:25.731663+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:25.731663+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:32.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:32.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20028, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.729183+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:25.729183+0000', 'last_peered': '2024-09-16T04:59:25.729183+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:25.729183+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:32.734 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:32.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20328, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.724368+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:25.724368+0000', 'last_peered': '2024-09-16T04:59:25.724368+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:25.724368+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:32.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:32.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20367, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727711+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:25.727711+0000', 'last_peered': '2024-09-16T04:59:25.727711+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:25.727711+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:32.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:32.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19993, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.723713+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:59:25.723713+0000', 'last_peered': '2024-09-16T04:59:25.723713+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:59:25.723713+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:32.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:32.735 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 22010, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:30.110711+0000', 'last_change': '2024-09-16T04:59:24.057372+0000', 'last_active': '2024-09-16T04:59:30.110711+0000', 'last_peered': '2024-09-16T04:59:30.110711+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:59:30.110711+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2810, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 717, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2809}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:32.735 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:33.120 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:33.121 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23187/18 objects degraded (128816.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":720,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1489518592,"bytes_avail":288420773888,"bytes_total":289910292480,"degraded_objects":23187,"degraded_total":18,"degraded_ratio":1288.1666666666667,"recovering_objects_per_sec":151,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1216,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:59:33.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:59:33.141 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:33.515 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:33.515 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:33.536 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-16T04:59:32.557098+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17556,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454608,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660912,"statfs":{"total":289910292480,"available":288420773888,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1216,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1216,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002379"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26525,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728055+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728055+0000","last_peered":"2024-09-16T04:59:25.728055+0000","last_clean":"2024-09-16T04:59:25.728055+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728055+0000","last_undegraded":"2024-09-16T04:59:25.728055+0000","last_fullsized":"2024-09-16T04:59:25.728055+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722484+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722484+0000","last_peered":"2024-09-16T04:59:25.722484+0000","last_clean":"2024-09-16T04:59:25.722484+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722484+0000","last_undegraded":"2024-09-16T04:59:25.722484+0000","last_fullsized":"2024-09-16T04:59:25.722484+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26843,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723968+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.723968+0000","last_peered":"2024-09-16T04:59:25.723968+0000","last_clean":"2024-09-16T04:59:25.723968+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.723968+0000","last_undegraded":"2024-09-16T04:59:25.723968+0000","last_fullsized":"2024-09-16T04:59:25.723968+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":22010,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:30.110711+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:30.110711+0000","last_peered":"2024-09-16T04:59:30.110711+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:30.110711+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2810,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17555,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593847,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473664,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898176,"statfs":{"total":96636764160,"available":96151732224,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:36.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:36.857 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:36.858 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:36.879 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-16T04:59:36.558032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18256,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442128,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673392,"statfs":{"total":289910292480,"available":288433553408,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1340,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.836498"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26525,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728055+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728055+0000","last_peered":"2024-09-16T04:59:25.728055+0000","last_clean":"2024-09-16T04:59:25.728055+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728055+0000","last_undegraded":"2024-09-16T04:59:25.728055+0000","last_fullsized":"2024-09-16T04:59:25.728055+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722484+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722484+0000","last_peered":"2024-09-16T04:59:25.722484+0000","last_clean":"2024-09-16T04:59:25.722484+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722484+0000","last_undegraded":"2024-09-16T04:59:25.722484+0000","last_fullsized":"2024-09-16T04:59:25.722484+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26843,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723968+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.723968+0000","last_peered":"2024-09-16T04:59:25.723968+0000","last_clean":"2024-09-16T04:59:25.723968+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.723968+0000","last_undegraded":"2024-09-16T04:59:25.723968+0000","last_fullsized":"2024-09-16T04:59:25.723968+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":23410,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:35.109268+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:35.109268+0000","last_peered":"2024-09-16T04:59:35.109268+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:35.109268+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1417,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18255,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593848,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461184,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910656,"statfs":{"total":96636764160,"available":96164511744,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:36.880 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:37.194 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:37.195 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:37.211 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-16T04:59:36.558032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18256,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442128,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673392,"statfs":{"total":289910292480,"available":288433553408,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1340,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.836498"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26525,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728055+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728055+0000","last_peered":"2024-09-16T04:59:25.728055+0000","last_clean":"2024-09-16T04:59:25.728055+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728055+0000","last_undegraded":"2024-09-16T04:59:25.728055+0000","last_fullsized":"2024-09-16T04:59:25.728055+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722484+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722484+0000","last_peered":"2024-09-16T04:59:25.722484+0000","last_clean":"2024-09-16T04:59:25.722484+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722484+0000","last_undegraded":"2024-09-16T04:59:25.722484+0000","last_fullsized":"2024-09-16T04:59:25.722484+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26843,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723968+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.723968+0000","last_peered":"2024-09-16T04:59:25.723968+0000","last_clean":"2024-09-16T04:59:25.723968+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.723968+0000","last_undegraded":"2024-09-16T04:59:25.723968+0000","last_fullsized":"2024-09-16T04:59:25.723968+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":23410,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:35.109268+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:35.109268+0000","last_peered":"2024-09-16T04:59:35.109268+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:35.109268+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1417,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18255,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593848,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461184,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910656,"statfs":{"total":96636764160,"available":96164511744,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:37.213 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:37.213 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20016, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727514+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:25.727514+0000', 'last_peered': '2024-09-16T04:59:25.727514+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:25.727514+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:37.213 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:37.213 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20466, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.731663+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:25.731663+0000', 'last_peered': '2024-09-16T04:59:25.731663+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:25.731663+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:37.213 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:37.213 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20028, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.729183+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:25.729183+0000', 'last_peered': '2024-09-16T04:59:25.729183+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:25.729183+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:37.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:37.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20328, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.724368+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:25.724368+0000', 'last_peered': '2024-09-16T04:59:25.724368+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:25.724368+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:37.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:37.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20367, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727711+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:25.727711+0000', 'last_peered': '2024-09-16T04:59:25.727711+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:25.727711+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:37.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:37.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19993, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.723713+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:59:25.723713+0000', 'last_peered': '2024-09-16T04:59:25.723713+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:59:25.723713+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:37.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:37.214 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 23410, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:35.109268+0000', 'last_change': '2024-09-16T04:59:24.057372+0000', 'last_active': '2024-09-16T04:59:35.109268+0000', 'last_peered': '2024-09-16T04:59:35.109268+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:59:35.109268+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2110, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1417, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2109}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:37.215 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:37.579 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:37.579 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22487/18 objects degraded (124927.778%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":725,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1476739072,"bytes_avail":288433553408,"bytes_total":289910292480,"degraded_objects":22487,"degraded_total":18,"degraded_ratio":1249.2777777777778,"recovering_objects_per_sec":123,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1340,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:59:37.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:59:37.600 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:37.957 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:37.958 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:37.975 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-16T04:59:36.558032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18256,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442128,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673392,"statfs":{"total":289910292480,"available":288433553408,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1340,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.836498"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26525,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728055+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728055+0000","last_peered":"2024-09-16T04:59:25.728055+0000","last_clean":"2024-09-16T04:59:25.728055+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728055+0000","last_undegraded":"2024-09-16T04:59:25.728055+0000","last_fullsized":"2024-09-16T04:59:25.728055+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722484+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722484+0000","last_peered":"2024-09-16T04:59:25.722484+0000","last_clean":"2024-09-16T04:59:25.722484+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722484+0000","last_undegraded":"2024-09-16T04:59:25.722484+0000","last_fullsized":"2024-09-16T04:59:25.722484+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26843,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723968+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.723968+0000","last_peered":"2024-09-16T04:59:25.723968+0000","last_clean":"2024-09-16T04:59:25.723968+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.723968+0000","last_undegraded":"2024-09-16T04:59:25.723968+0000","last_fullsized":"2024-09-16T04:59:25.723968+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":23410,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:35.109268+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:35.109268+0000","last_peered":"2024-09-16T04:59:35.109268+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:35.109268+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1417,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2109}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18255,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593848,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461184,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910656,"statfs":{"total":96636764160,"available":96164511744,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:40.980 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:41.321 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:41.321 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:41.340 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-16T04:59:40.559197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18635,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446288,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281669232,"statfs":{"total":289910292480,"available":288429293568,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1719,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1719,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003213"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26843,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723968+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.723968+0000","last_peered":"2024-09-16T04:59:25.723968+0000","last_clean":"2024-09-16T04:59:25.723968+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.723968+0000","last_undegraded":"2024-09-16T04:59:25.723968+0000","last_fullsized":"2024-09-16T04:59:25.723968+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":24168,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:40.114651+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:40.114651+0000","last_peered":"2024-09-16T04:59:40.114651+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:40.114651+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1796,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1730}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18634,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593849,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465344,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906496,"statfs":{"total":96636764160,"available":96160251904,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:41.341 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:41.653 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:41.654 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:41.674 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-16T04:59:40.559197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18635,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446288,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281669232,"statfs":{"total":289910292480,"available":288429293568,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1719,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1719,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003213"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26843,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723968+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.723968+0000","last_peered":"2024-09-16T04:59:25.723968+0000","last_clean":"2024-09-16T04:59:25.723968+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.723968+0000","last_undegraded":"2024-09-16T04:59:25.723968+0000","last_fullsized":"2024-09-16T04:59:25.723968+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":24168,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:40.114651+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:40.114651+0000","last_peered":"2024-09-16T04:59:40.114651+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:40.114651+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1796,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1730}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18634,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593849,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465344,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906496,"statfs":{"total":96636764160,"available":96160251904,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:41.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:41.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20016, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727514+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:25.727514+0000', 'last_peered': '2024-09-16T04:59:25.727514+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:25.727514+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:41.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:41.676 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20466, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.731663+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:25.731663+0000', 'last_peered': '2024-09-16T04:59:25.731663+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:25.731663+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:41.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:41.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20028, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.729183+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:25.729183+0000', 'last_peered': '2024-09-16T04:59:25.729183+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:25.729183+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:41.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:41.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20328, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.724368+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:25.724368+0000', 'last_peered': '2024-09-16T04:59:25.724368+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:25.724368+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:41.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:41.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20367, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727711+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:25.727711+0000', 'last_peered': '2024-09-16T04:59:25.727711+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:25.727711+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:41.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:41.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19993, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.723713+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:59:25.723713+0000', 'last_peered': '2024-09-16T04:59:25.723713+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:59:25.723713+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:41.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:41.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 24168, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:40.114651+0000', 'last_change': '2024-09-16T04:59:24.057372+0000', 'last_active': '2024-09-16T04:59:40.114651+0000', 'last_peered': '2024-09-16T04:59:40.114651+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:59:40.114651+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1731, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1796, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1730}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:41.678 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:42.097 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:42.097 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22108/18 objects degraded (122822.222%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":729,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1480998912,"bytes_avail":288429293568,"bytes_total":289910292480,"degraded_objects":22108,"degraded_total":18,"degraded_ratio":1228.2222222222222,"recovering_objects_per_sec":143,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1719,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:59:42.122 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:59:42.122 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:42.479 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:42.498 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-16T04:59:40.559197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18635,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446288,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281669232,"statfs":{"total":289910292480,"available":288429293568,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1719,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1719,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003213"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26843,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723968+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.723968+0000","last_peered":"2024-09-16T04:59:25.723968+0000","last_clean":"2024-09-16T04:59:25.723968+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.723968+0000","last_undegraded":"2024-09-16T04:59:25.723968+0000","last_fullsized":"2024-09-16T04:59:25.723968+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":24168,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:40.114651+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:40.114651+0000","last_peered":"2024-09-16T04:59:40.114651+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:40.114651+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1796,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1730}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18634,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593849,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465344,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906496,"statfs":{"total":96636764160,"available":96160251904,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:45.500 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:45.838 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:45.839 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:45.857 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-16T04:59:44.560578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18635,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446288,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281669232,"statfs":{"total":289910292480,"available":288429293568,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1079,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003479"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26843,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723968+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.723968+0000","last_peered":"2024-09-16T04:59:25.723968+0000","last_clean":"2024-09-16T04:59:25.723968+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.723968+0000","last_undegraded":"2024-09-16T04:59:25.723968+0000","last_fullsized":"2024-09-16T04:59:25.723968+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":24168,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:40.114651+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:40.114651+0000","last_peered":"2024-09-16T04:59:40.114651+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:40.114651+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1796,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1730}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18634,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593849,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465344,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906496,"statfs":{"total":96636764160,"available":96160251904,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:45.859 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:46.212 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:46.212 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:46.231 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-16T04:59:44.560578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18635,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446288,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281669232,"statfs":{"total":289910292480,"available":288429293568,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1079,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003479"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26843,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723968+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.723968+0000","last_peered":"2024-09-16T04:59:25.723968+0000","last_clean":"2024-09-16T04:59:25.723968+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.723968+0000","last_undegraded":"2024-09-16T04:59:25.723968+0000","last_fullsized":"2024-09-16T04:59:25.723968+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":24168,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:40.114651+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:40.114651+0000","last_peered":"2024-09-16T04:59:40.114651+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:40.114651+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1731,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1796,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1730}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18634,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593849,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465344,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906496,"statfs":{"total":96636764160,"available":96160251904,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:46.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:46.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20016, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727514+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:25.727514+0000', 'last_peered': '2024-09-16T04:59:25.727514+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:25.727514+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:46.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:46.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20466, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.731663+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:25.731663+0000', 'last_peered': '2024-09-16T04:59:25.731663+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:25.731663+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:46.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:46.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20028, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.729183+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:25.729183+0000', 'last_peered': '2024-09-16T04:59:25.729183+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:25.729183+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:46.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:46.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20328, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.724368+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:25.724368+0000', 'last_peered': '2024-09-16T04:59:25.724368+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:25.724368+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:46.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:46.234 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20367, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727711+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:25.727711+0000', 'last_peered': '2024-09-16T04:59:25.727711+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:25.727711+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:46.234 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:46.234 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19993, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.723713+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:59:25.723713+0000', 'last_peered': '2024-09-16T04:59:25.723713+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:59:25.723713+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:46.234 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:46.234 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 24168, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:40.114651+0000', 'last_change': '2024-09-16T04:59:24.057372+0000', 'last_active': '2024-09-16T04:59:40.114651+0000', 'last_peered': '2024-09-16T04:59:40.114651+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:59:40.114651+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1731, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1796, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1730}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:46.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:46.601 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:46.602 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22108/18 objects degraded (122822.222%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":734,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1480998912,"bytes_avail":288429293568,"bytes_total":289910292480,"degraded_objects":22108,"degraded_total":18,"degraded_ratio":1228.2222222222222,"recovering_objects_per_sec":89,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1079,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:59:46.621 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:59:46.622 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:46.924 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:46.942 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-16T04:59:46.561280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19043,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450448,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665072,"statfs":{"total":289910292480,"available":288425033728,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1487,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1487,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003845"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26843,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723968+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.723968+0000","last_peered":"2024-09-16T04:59:25.723968+0000","last_clean":"2024-09-16T04:59:25.723968+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.723968+0000","last_undegraded":"2024-09-16T04:59:25.723968+0000","last_fullsized":"2024-09-16T04:59:25.723968+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":24984,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:45.111954+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:45.111954+0000","last_peered":"2024-09-16T04:59:45.111954+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:45.111954+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2204,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1322}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19042,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593850,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469504,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902336,"statfs":{"total":96636764160,"available":96155992064,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:49.945 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:50.268 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:50.269 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:50.287 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-09-16T04:59:48.561806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19043,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450448,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665072,"statfs":{"total":289910292480,"available":288425033728,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-787,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003774"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26843,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723968+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.723968+0000","last_peered":"2024-09-16T04:59:25.723968+0000","last_clean":"2024-09-16T04:59:25.723968+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.723968+0000","last_undegraded":"2024-09-16T04:59:25.723968+0000","last_fullsized":"2024-09-16T04:59:25.723968+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":24984,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:45.111954+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:45.111954+0000","last_peered":"2024-09-16T04:59:45.111954+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:45.111954+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2204,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1322}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21700,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19042,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593850,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469504,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902336,"statfs":{"total":96636764160,"available":96155992064,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:50.290 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:50.592 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:50.592 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:50.614 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-16T04:59:50.562404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19536,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454608,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660912,"statfs":{"total":289910292480,"available":288420773888,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1280,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1280,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003875"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26844,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.724043+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.724043+0000","last_peered":"2024-09-16T04:59:25.724043+0000","last_clean":"2024-09-16T04:59:25.724043+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.724043+0000","last_undegraded":"2024-09-16T04:59:25.724043+0000","last_fullsized":"2024-09-16T04:59:25.724043+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:52.952809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":25970,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:50.115759+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:50.115759+0000","last_peered":"2024-09-16T04:59:50.115759+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:50.115759+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2697,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19535,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593851,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473664,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898176,"statfs":{"total":96636764160,"available":96151732224,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:50.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:50.615 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20016, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727514+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:25.727514+0000', 'last_peered': '2024-09-16T04:59:25.727514+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:25.727514+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:50.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:50.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20466, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.731663+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:25.731663+0000', 'last_peered': '2024-09-16T04:59:25.731663+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:25.731663+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:50.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:50.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20028, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.729183+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:25.729183+0000', 'last_peered': '2024-09-16T04:59:25.729183+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:25.729183+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:50.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:50.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20328, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.724368+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:25.724368+0000', 'last_peered': '2024-09-16T04:59:25.724368+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:25.724368+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:50.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:50.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20367, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727711+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:25.727711+0000', 'last_peered': '2024-09-16T04:59:25.727711+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:25.727711+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:50.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:50.616 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19993, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.723713+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:59:25.723713+0000', 'last_peered': '2024-09-16T04:59:25.723713+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:59:25.723713+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:50.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:50.617 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 25970, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:50.115759+0000', 'last_change': '2024-09-16T04:59:24.057372+0000', 'last_active': '2024-09-16T04:59:50.115759+0000', 'last_peered': '2024-09-16T04:59:50.115759+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:59:50.115759+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 830, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2697, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -829}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:50.617 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:51.004 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:51.005 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21207/18 objects degraded (117816.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":738,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1489518592,"bytes_avail":288420773888,"bytes_total":289910292480,"degraded_objects":21207,"degraded_total":18,"degraded_ratio":1178.1666666666667,"recovering_objects_per_sec":106,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1280,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:59:51.030 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:59:51.030 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:51.343 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:51.360 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-16T04:59:50.562404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19536,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454608,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660912,"statfs":{"total":289910292480,"available":288420773888,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1280,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1280,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003875"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26844,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.724043+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.724043+0000","last_peered":"2024-09-16T04:59:25.724043+0000","last_clean":"2024-09-16T04:59:25.724043+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.724043+0000","last_undegraded":"2024-09-16T04:59:25.724043+0000","last_fullsized":"2024-09-16T04:59:25.724043+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:52.952809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":25970,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:50.115759+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:50.115759+0000","last_peered":"2024-09-16T04:59:50.115759+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:50.115759+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2697,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19535,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593851,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473664,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898176,"statfs":{"total":96636764160,"available":96151732224,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:54.362 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:54.686 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:54.686 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:54.706 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-16T04:59:54.563751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19536,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454608,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660912,"statfs":{"total":289910292480,"available":288420773888,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":901,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003560"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26844,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.724043+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.724043+0000","last_peered":"2024-09-16T04:59:25.724043+0000","last_clean":"2024-09-16T04:59:25.724043+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.724043+0000","last_undegraded":"2024-09-16T04:59:25.724043+0000","last_fullsized":"2024-09-16T04:59:25.724043+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:52.952809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":25970,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:50.115759+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:50.115759+0000","last_peered":"2024-09-16T04:59:50.115759+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:50.115759+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2697,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19535,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593851,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473664,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898176,"statfs":{"total":96636764160,"available":96151732224,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:54.708 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:55.017 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:55.039 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-16T04:59:54.563751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19536,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454608,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660912,"statfs":{"total":289910292480,"available":288420773888,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":901,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003560"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26844,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.724043+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.724043+0000","last_peered":"2024-09-16T04:59:25.724043+0000","last_clean":"2024-09-16T04:59:25.724043+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.724043+0000","last_undegraded":"2024-09-16T04:59:25.724043+0000","last_fullsized":"2024-09-16T04:59:25.724043+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:52.952809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":25970,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:50.115759+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:50.115759+0000","last_peered":"2024-09-16T04:59:50.115759+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:50.115759+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2697,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19535,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593851,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473664,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898176,"statfs":{"total":96636764160,"available":96151732224,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:55.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:55.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20016, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727514+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:25.727514+0000', 'last_peered': '2024-09-16T04:59:25.727514+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:25.727514+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:55.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:55.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20466, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.731663+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:25.731663+0000', 'last_peered': '2024-09-16T04:59:25.731663+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:25.731663+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:55.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:55.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20028, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.729183+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:25.729183+0000', 'last_peered': '2024-09-16T04:59:25.729183+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:25.729183+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:55.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:55.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20328, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.724368+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:25.724368+0000', 'last_peered': '2024-09-16T04:59:25.724368+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:25.724368+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:55.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:55.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20367, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727711+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:25.727711+0000', 'last_peered': '2024-09-16T04:59:25.727711+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:25.727711+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:55.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:55.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19993, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.723713+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:59:25.723713+0000', 'last_peered': '2024-09-16T04:59:25.723713+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:59:25.723713+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:55.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:55.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 25970, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:50.115759+0000', 'last_change': '2024-09-16T04:59:24.057372+0000', 'last_active': '2024-09-16T04:59:50.115759+0000', 'last_peered': '2024-09-16T04:59:50.115759+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:59:50.115759+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 830, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2697, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -829}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:55.042 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:55.433 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:55.433 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21207/18 objects degraded (117816.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":743,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1489518592,"bytes_avail":288420773888,"bytes_total":289910292480,"degraded_objects":21207,"degraded_total":18,"degraded_ratio":1178.1666666666667,"recovering_objects_per_sec":75,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":901,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T04:59:55.452 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T04:59:55.452 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:55.771 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:55.772 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:55.789 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-16T04:59:54.563751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19536,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454608,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660912,"statfs":{"total":289910292480,"available":288420773888,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":901,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003560"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20328,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.724368+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.724368+0000","last_peered":"2024-09-16T04:59:25.724368+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.724368+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26844,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.724043+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.724043+0000","last_peered":"2024-09-16T04:59:25.724043+0000","last_clean":"2024-09-16T04:59:25.724043+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.724043+0000","last_undegraded":"2024-09-16T04:59:25.724043+0000","last_fullsized":"2024-09-16T04:59:25.724043+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:52.952809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":25970,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:50.115759+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:50.115759+0000","last_peered":"2024-09-16T04:59:50.115759+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:50.115759+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2697,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19535,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593851,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473664,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898176,"statfs":{"total":96636764160,"available":96151732224,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:58.792 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:59.163 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:59.164 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:59.188 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-16T04:59:58.565208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20136,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442128,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673392,"statfs":{"total":289910292480,"available":288433553408,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002804"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20329,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.725904+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.725904+0000","last_peered":"2024-09-16T04:59:25.725904+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.725904+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26844,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.724043+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.724043+0000","last_peered":"2024-09-16T04:59:25.724043+0000","last_clean":"2024-09-16T04:59:25.724043+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.724043+0000","last_undegraded":"2024-09-16T04:59:25.724043+0000","last_fullsized":"2024-09-16T04:59:25.724043+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:52.952809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":27171,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:55.121136+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:55.121136+0000","last_peered":"2024-09-16T04:59:55.121136+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:55.121136+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20135,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461184,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910656,"statfs":{"total":96636764160,"available":96164511744,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:59.190 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T04:59:59.540 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:59.541 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T04:59:59.566 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-16T04:59:58.565208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20136,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442128,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673392,"statfs":{"total":289910292480,"available":288433553408,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002804"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20329,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.725904+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.725904+0000","last_peered":"2024-09-16T04:59:25.725904+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.725904+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26844,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.724043+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.724043+0000","last_peered":"2024-09-16T04:59:25.724043+0000","last_clean":"2024-09-16T04:59:25.724043+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.724043+0000","last_undegraded":"2024-09-16T04:59:25.724043+0000","last_fullsized":"2024-09-16T04:59:25.724043+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:52.952809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":27171,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:55.121136+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:55.121136+0000","last_peered":"2024-09-16T04:59:55.121136+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:55.121136+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20135,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461184,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910656,"statfs":{"total":96636764160,"available":96164511744,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T04:59:59.567 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T04:59:59.567 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20016, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727514+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:25.727514+0000', 'last_peered': '2024-09-16T04:59:25.727514+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:25.727514+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:59.567 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T04:59:59.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20466, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.731663+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:25.731663+0000', 'last_peered': '2024-09-16T04:59:25.731663+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:25.731663+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:59.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T04:59:59.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20028, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.729183+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:25.729183+0000', 'last_peered': '2024-09-16T04:59:25.729183+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:25.729183+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:59.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T04:59:59.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20329, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.725904+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:25.725904+0000', 'last_peered': '2024-09-16T04:59:25.725904+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:25.725904+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:59.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T04:59:59.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20367, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.727711+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:25.727711+0000', 'last_peered': '2024-09-16T04:59:25.727711+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:25.727711+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:59.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T04:59:59.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 19993, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:25.723713+0000', 'last_change': '2024-09-16T04:49:38.182176+0000', 'last_active': '2024-09-16T04:59:25.723713+0000', 'last_peered': '2024-09-16T04:59:25.723713+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T04:59:25.723713+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3408, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T04:59:59.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-16T04:59:59.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "51'20416", 'reported_seq': 27171, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:55.121136+0000', 'last_change': '2024-09-16T04:59:24.057372+0000', 'last_active': '2024-09-16T04:59:55.121136+0000', 'last_peered': '2024-09-16T04:59:55.121136+0000', 'last_clean': '2024-09-16T04:49:35.635773+0000', 'last_became_active': '2024-09-16T04:49:38.231143+0000', 'last_became_peered': '2024-09-16T04:49:38.231143+0000', 'last_unstale': '2024-09-16T04:59:55.121136+0000', 'last_undegraded': '2024-09-16T04:49:37.759636+0000', 'last_fullsized': '2024-09-16T04:49:37.647472+0000', 'mapping_epoch': 44, 'log_start': "47'10400", 'ondisk_log_start': "47'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3269", 'last_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:27.641254+0000', 'objects_scrubbed': 3220, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1411, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 230, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17185, 'num_write_kb': 27932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3297, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -229}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T04:59:59.569 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T04:59:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T04:59:59.995 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20607/18 objects degraded (114483.333%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":747,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1476739072,"bytes_avail":288433553408,"bytes_total":289910292480,"degraded_objects":20607,"degraded_total":18,"degraded_ratio":1144.8333333333333,"recovering_objects_per_sec":74,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":600,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:00.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:00.019 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:00.328 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:00.329 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:00.346 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-16T04:59:58.565208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20136,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442128,"kb_used_data":164240,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673392,"statfs":{"total":289910292480,"available":288433553408,"internally_reserved":0,"allocated":168181760,"data_stored":167215869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-600,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002804"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26526,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.728124+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:25.728124+0000","last_peered":"2024-09-16T04:59:25.728124+0000","last_clean":"2024-09-16T04:59:25.728124+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:25.728124+0000","last_undegraded":"2024-09-16T04:59:25.728124+0000","last_fullsized":"2024-09-16T04:59:25.728124+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723688+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:25.723688+0000","last_peered":"2024-09-16T04:59:25.723688+0000","last_clean":"2024-09-16T04:59:25.723688+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:25.723688+0000","last_undegraded":"2024-09-16T04:59:25.723688+0000","last_fullsized":"2024-09-16T04:59:25.723688+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20016,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727514+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:25.727514+0000","last_peered":"2024-09-16T04:59:25.727514+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:25.727514+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20433,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727216+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:25.727216+0000","last_peered":"2024-09-16T04:59:25.727216+0000","last_clean":"2024-09-16T04:59:25.727216+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:25.727216+0000","last_undegraded":"2024-09-16T04:59:25.727216+0000","last_fullsized":"2024-09-16T04:59:25.727216+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20382,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727843+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:25.727843+0000","last_peered":"2024-09-16T04:59:25.727843+0000","last_clean":"2024-09-16T04:59:25.727843+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:25.727843+0000","last_undegraded":"2024-09-16T04:59:25.727843+0000","last_fullsized":"2024-09-16T04:59:25.727843+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20466,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.731663+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:25.731663+0000","last_peered":"2024-09-16T04:59:25.731663+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:25.731663+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.729183+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:25.729183+0000","last_peered":"2024-09-16T04:59:25.729183+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:25.729183+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20251,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727503+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:25.727503+0000","last_peered":"2024-09-16T04:59:25.727503+0000","last_clean":"2024-09-16T04:59:25.727503+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:25.727503+0000","last_undegraded":"2024-09-16T04:59:25.727503+0000","last_fullsized":"2024-09-16T04:59:25.727503+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722582+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:25.722582+0000","last_peered":"2024-09-16T04:59:25.722582+0000","last_clean":"2024-09-16T04:59:25.722582+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:25.722582+0000","last_undegraded":"2024-09-16T04:59:25.722582+0000","last_fullsized":"2024-09-16T04:59:25.722582+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20329,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.725904+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:25.725904+0000","last_peered":"2024-09-16T04:59:25.725904+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:25.725904+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723717+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:25.723717+0000","last_peered":"2024-09-16T04:59:25.723717+0000","last_clean":"2024-09-16T04:59:25.723717+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:25.723717+0000","last_undegraded":"2024-09-16T04:59:25.723717+0000","last_fullsized":"2024-09-16T04:59:25.723717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20367,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.727711+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:25.727711+0000","last_peered":"2024-09-16T04:59:25.727711+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:25.727711+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.727490+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:25.727490+0000","last_peered":"2024-09-16T04:59:25.727490+0000","last_clean":"2024-09-16T04:59:25.727490+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:25.727490+0000","last_undegraded":"2024-09-16T04:59:25.727490+0000","last_fullsized":"2024-09-16T04:59:25.727490+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26844,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.724043+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:25.724043+0000","last_peered":"2024-09-16T04:59:25.724043+0000","last_clean":"2024-09-16T04:59:25.724043+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:25.724043+0000","last_undegraded":"2024-09-16T04:59:25.724043+0000","last_fullsized":"2024-09-16T04:59:25.724043+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:52.952809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722696+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:25.722696+0000","last_peered":"2024-09-16T04:59:25.722696+0000","last_clean":"2024-09-16T04:59:25.722696+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:25.722696+0000","last_undegraded":"2024-09-16T04:59:25.722696+0000","last_fullsized":"2024-09-16T04:59:25.722696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26854,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723422+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:25.723422+0000","last_peered":"2024-09-16T04:59:25.723422+0000","last_clean":"2024-09-16T04:59:25.723422+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:25.723422+0000","last_undegraded":"2024-09-16T04:59:25.723422+0000","last_fullsized":"2024-09-16T04:59:25.723422+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19993,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:25.723713+0000","last_change":"2024-09-16T04:49:38.182176+0000","last_active":"2024-09-16T04:59:25.723713+0000","last_peered":"2024-09-16T04:59:25.723713+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T04:59:25.723713+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723017+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:25.723017+0000","last_peered":"2024-09-16T04:59:25.723017+0000","last_clean":"2024-09-16T04:59:25.723017+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:25.723017+0000","last_undegraded":"2024-09-16T04:59:25.723017+0000","last_fullsized":"2024-09-16T04:59:25.723017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20658,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.904348+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:25.904348+0000","last_peered":"2024-09-16T04:59:25.904348+0000","last_clean":"2024-09-16T04:59:25.904348+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:25.904348+0000","last_undegraded":"2024-09-16T04:59:25.904348+0000","last_fullsized":"2024-09-16T04:59:25.904348+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.729343+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:25.729343+0000","last_peered":"2024-09-16T04:59:25.729343+0000","last_clean":"2024-09-16T04:59:25.729343+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:25.729343+0000","last_undegraded":"2024-09-16T04:59:25.729343+0000","last_fullsized":"2024-09-16T04:59:25.729343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27011,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723009+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:25.723009+0000","last_peered":"2024-09-16T04:59:25.723009+0000","last_clean":"2024-09-16T04:59:25.723009+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:25.723009+0000","last_undegraded":"2024-09-16T04:59:25.723009+0000","last_fullsized":"2024-09-16T04:59:25.723009+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.722598+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:25.722598+0000","last_peered":"2024-09-16T04:59:25.722598+0000","last_clean":"2024-09-16T04:59:25.722598+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:25.722598+0000","last_undegraded":"2024-09-16T04:59:25.722598+0000","last_fullsized":"2024-09-16T04:59:25.722598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T04:59:25.723012+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:25.723012+0000","last_peered":"2024-09-16T04:59:25.723012+0000","last_clean":"2024-09-16T04:59:25.723012+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:25.723012+0000","last_undegraded":"2024-09-16T04:59:25.723012+0000","last_fullsized":"2024-09-16T04:59:25.723012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":27171,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:55.121136+0000","last_change":"2024-09-16T04:59:24.057372+0000","last_active":"2024-09-16T04:59:55.121136+0000","last_peered":"2024-09-16T04:59:55.121136+0000","last_clean":"2024-09-16T04:49:35.635773+0000","last_became_active":"2024-09-16T04:49:38.231143+0000","last_became_peered":"2024-09-16T04:49:38.231143+0000","last_unstale":"2024-09-16T04:59:55.121136+0000","last_undegraded":"2024-09-16T04:49:37.759636+0000","last_fullsized":"2024-09-16T04:49:37.647472+0000","mapping_epoch":44,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20607,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20135,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166793216,"data_stored":166747989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493196,"kb_used_data":82484,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878644,"statfs":{"total":96636764160,"available":96131731456,"internally_reserved":0,"allocated":84463616,"data_stored":84153249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461184,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910656,"statfs":{"total":96636764160,"available":96164511744,"internally_reserved":0,"allocated":507904,"data_stored":162747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487748,"kb_used_data":81260,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884092,"statfs":{"total":96636764160,"available":96137310208,"internally_reserved":0,"allocated":83210240,"data_stored":82899873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:03.349 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:03.721 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:03.742 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-16T05:00:02.566387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20826,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450496,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665024,"statfs":{"total":289910292480,"available":288424984576,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1290,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002635"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20133,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713164+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713164+0000","last_peered":"2024-09-16T04:59:57.713164+0000","last_clean":"2024-09-16T04:59:57.713164+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713164+0000","last_undegraded":"2024-09-16T04:59:57.713164+0000","last_fullsized":"2024-09-16T04:59:57.713164+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.713048+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.713048+0000","last_peered":"2024-09-16T04:59:57.713048+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.713048+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20437,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712588+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712588+0000","last_peered":"2024-09-16T04:59:57.712588+0000","last_clean":"2024-09-16T04:59:57.712588+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712588+0000","last_undegraded":"2024-09-16T04:59:57.712588+0000","last_fullsized":"2024-09-16T04:59:57.712588+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20255,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715401+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715401+0000","last_peered":"2024-09-16T04:59:57.715401+0000","last_clean":"2024-09-16T04:59:57.715401+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715401+0000","last_undegraded":"2024-09-16T04:59:57.715401+0000","last_fullsized":"2024-09-16T04:59:57.715401+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713422+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713422+0000","last_peered":"2024-09-16T04:59:57.713422+0000","last_clean":"2024-09-16T04:59:57.713422+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713422+0000","last_undegraded":"2024-09-16T04:59:57.713422+0000","last_fullsized":"2024-09-16T04:59:57.713422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20918,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:00.027872+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:00.027872+0000","last_peered":"2024-09-16T05:00:00.027872+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:00.027872+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2948,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":462,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2947}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714239+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714239+0000","last_peered":"2024-09-16T04:59:57.714239+0000","last_clean":"2024-09-16T04:59:57.714239+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714239+0000","last_undegraded":"2024-09-16T04:59:57.714239+0000","last_fullsized":"2024-09-16T04:59:57.714239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714366+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714366+0000","last_peered":"2024-09-16T04:59:57.714366+0000","last_clean":"2024-09-16T04:59:57.714366+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714366+0000","last_undegraded":"2024-09-16T04:59:57.714366+0000","last_fullsized":"2024-09-16T04:59:57.714366+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20825,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469520,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902320,"statfs":{"total":96636764160,"available":96155975680,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:03.743 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:04.125 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:04.147 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-16T05:00:02.566387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20826,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450496,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665024,"statfs":{"total":289910292480,"available":288424984576,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1290,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002635"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20133,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713164+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713164+0000","last_peered":"2024-09-16T04:59:57.713164+0000","last_clean":"2024-09-16T04:59:57.713164+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713164+0000","last_undegraded":"2024-09-16T04:59:57.713164+0000","last_fullsized":"2024-09-16T04:59:57.713164+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.713048+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.713048+0000","last_peered":"2024-09-16T04:59:57.713048+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.713048+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20437,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712588+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712588+0000","last_peered":"2024-09-16T04:59:57.712588+0000","last_clean":"2024-09-16T04:59:57.712588+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712588+0000","last_undegraded":"2024-09-16T04:59:57.712588+0000","last_fullsized":"2024-09-16T04:59:57.712588+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20255,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715401+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715401+0000","last_peered":"2024-09-16T04:59:57.715401+0000","last_clean":"2024-09-16T04:59:57.715401+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715401+0000","last_undegraded":"2024-09-16T04:59:57.715401+0000","last_fullsized":"2024-09-16T04:59:57.715401+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713422+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713422+0000","last_peered":"2024-09-16T04:59:57.713422+0000","last_clean":"2024-09-16T04:59:57.713422+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713422+0000","last_undegraded":"2024-09-16T04:59:57.713422+0000","last_fullsized":"2024-09-16T04:59:57.713422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20918,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:00.027872+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:00.027872+0000","last_peered":"2024-09-16T05:00:00.027872+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:00.027872+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2948,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":462,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2947}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714239+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714239+0000","last_peered":"2024-09-16T04:59:57.714239+0000","last_clean":"2024-09-16T04:59:57.714239+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714239+0000","last_undegraded":"2024-09-16T04:59:57.714239+0000","last_fullsized":"2024-09-16T04:59:57.714239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714366+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714366+0000","last_peered":"2024-09-16T04:59:57.714366+0000","last_clean":"2024-09-16T04:59:57.714366+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714366+0000","last_undegraded":"2024-09-16T04:59:57.714366+0000","last_fullsized":"2024-09-16T04:59:57.714366+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20825,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469520,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902320,"statfs":{"total":96636764160,"available":96155975680,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:04.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:00:04.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20020, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.713048+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:57.713048+0000', 'last_peered': '2024-09-16T04:59:57.713048+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:57.713048+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:04.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:00:04.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20470, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.715414+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:57.715414+0000', 'last_peered': '2024-09-16T04:59:57.715414+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:57.715414+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:04.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:00:04.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20032, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.715741+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:57.715741+0000', 'last_peered': '2024-09-16T04:59:57.715741+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:57.715741+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:04.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:00:04.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20333, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.714952+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:57.714952+0000', 'last_peered': '2024-09-16T04:59:57.714952+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:57.714952+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:04.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T05:00:04.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20372, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.718401+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:57.718401+0000', 'last_peered': '2024-09-16T04:59:57.718401+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:57.718401+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:04.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T05:00:04.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 20918, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:00.027872+0000', 'last_change': '2024-09-16T04:59:56.595445+0000', 'last_active': '2024-09-16T05:00:00.027872+0000', 'last_peered': '2024-09-16T05:00:00.027872+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T05:00:00.027872+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2948, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 462, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2947}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:04.151 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:00:04.551 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:04.552 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19917/18 objects degraded (110650.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":752,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1485307904,"bytes_avail":288424984576,"bytes_total":289910292480,"degraded_objects":19917,"degraded_total":18,"degraded_ratio":1106.5,"recovering_objects_per_sec":161,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1290,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:04.575 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:04.575 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:04.908 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:04.909 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:04.928 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-09-16T05:00:04.567097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20826,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450496,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281665024,"statfs":{"total":289910292480,"available":288424984576,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1316998145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-690,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":690,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002654"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20133,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713164+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713164+0000","last_peered":"2024-09-16T04:59:57.713164+0000","last_clean":"2024-09-16T04:59:57.713164+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713164+0000","last_undegraded":"2024-09-16T04:59:57.713164+0000","last_fullsized":"2024-09-16T04:59:57.713164+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.713048+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.713048+0000","last_peered":"2024-09-16T04:59:57.713048+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.713048+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20437,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712588+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712588+0000","last_peered":"2024-09-16T04:59:57.712588+0000","last_clean":"2024-09-16T04:59:57.712588+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712588+0000","last_undegraded":"2024-09-16T04:59:57.712588+0000","last_fullsized":"2024-09-16T04:59:57.712588+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20255,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715401+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715401+0000","last_peered":"2024-09-16T04:59:57.715401+0000","last_clean":"2024-09-16T04:59:57.715401+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715401+0000","last_undegraded":"2024-09-16T04:59:57.715401+0000","last_fullsized":"2024-09-16T04:59:57.715401+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713422+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713422+0000","last_peered":"2024-09-16T04:59:57.713422+0000","last_clean":"2024-09-16T04:59:57.713422+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713422+0000","last_undegraded":"2024-09-16T04:59:57.713422+0000","last_fullsized":"2024-09-16T04:59:57.713422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20918,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:00.027872+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:00.027872+0000","last_peered":"2024-09-16T05:00:00.027872+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:00.027872+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2948,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":462,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2947}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714239+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714239+0000","last_peered":"2024-09-16T04:59:57.714239+0000","last_clean":"2024-09-16T04:59:57.714239+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714239+0000","last_undegraded":"2024-09-16T04:59:57.714239+0000","last_fullsized":"2024-09-16T04:59:57.714239+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714366+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714366+0000","last_peered":"2024-09-16T04:59:57.714366+0000","last_clean":"2024-09-16T04:59:57.714366+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714366+0000","last_undegraded":"2024-09-16T04:59:57.714366+0000","last_fullsized":"2024-09-16T04:59:57.714366+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20825,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469520,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93902320,"statfs":{"total":96636764160,"available":96155975680,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":480243374},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:07.931 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:08.267 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:08.286 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-16T05:00:06.567424+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21597,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454652,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660868,"statfs":{"total":289910292480,"available":288420728832,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.864639"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20255,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715401+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715401+0000","last_peered":"2024-09-16T04:59:57.715401+0000","last_clean":"2024-09-16T04:59:57.715401+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715401+0000","last_undegraded":"2024-09-16T04:59:57.715401+0000","last_fullsized":"2024-09-16T04:59:57.715401+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":22460,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:05.031401+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:05.031401+0000","last_peered":"2024-09-16T05:00:05.031401+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:05.031401+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1233,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2176}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714366+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714366+0000","last_peered":"2024-09-16T04:59:57.714366+0000","last_clean":"2024-09-16T04:59:57.714366+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714366+0000","last_undegraded":"2024-09-16T04:59:57.714366+0000","last_fullsized":"2024-09-16T04:59:57.714366+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21596,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473676,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898164,"statfs":{"total":96636764160,"available":96151719936,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:08.287 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:08.702 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-16T05:00:08.567872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21597,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454652,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660868,"statfs":{"total":289910292480,"available":288420728832,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.859466"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20255,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715401+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715401+0000","last_peered":"2024-09-16T04:59:57.715401+0000","last_clean":"2024-09-16T04:59:57.715401+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715401+0000","last_undegraded":"2024-09-16T04:59:57.715401+0000","last_fullsized":"2024-09-16T04:59:57.715401+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":22460,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:05.031401+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:05.031401+0000","last_peered":"2024-09-16T05:00:05.031401+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:05.031401+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1233,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2176}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714366+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714366+0000","last_peered":"2024-09-16T04:59:57.714366+0000","last_clean":"2024-09-16T04:59:57.714366+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714366+0000","last_undegraded":"2024-09-16T04:59:57.714366+0000","last_fullsized":"2024-09-16T04:59:57.714366+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21596,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473676,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898164,"statfs":{"total":96636764160,"available":96151719936,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:08.703 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:00:08.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20021, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.718009+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:57.718009+0000', 'last_peered': '2024-09-16T04:59:57.718009+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:57.718009+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:08.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:00:08.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20470, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.715414+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:57.715414+0000', 'last_peered': '2024-09-16T04:59:57.715414+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:57.715414+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:08.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:00:08.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20032, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.715741+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:57.715741+0000', 'last_peered': '2024-09-16T04:59:57.715741+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:57.715741+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:08.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:00:08.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20333, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.714952+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:57.714952+0000', 'last_peered': '2024-09-16T04:59:57.714952+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:57.714952+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:08.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T05:00:08.704 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20372, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.718401+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:57.718401+0000', 'last_peered': '2024-09-16T04:59:57.718401+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:57.718401+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:08.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T05:00:08.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 22460, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:05.031401+0000', 'last_change': '2024-09-16T04:59:56.595445+0000', 'last_active': '2024-09-16T05:00:05.031401+0000', 'last_peered': '2024-09-16T05:00:05.031401+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T05:00:05.031401+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2177, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1233, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2176}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:08.705 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:00:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:09.086 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19146/18 objects degraded (106366.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":756,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1489563648,"bytes_avail":288420728832,"bytes_total":289910292480,"degraded_objects":19146,"degraded_total":18,"degraded_ratio":1063.6666666666667,"recovering_objects_per_sec":134,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1461,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:09.105 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:09.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:09.425 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:09.426 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:09.447 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-16T05:00:08.567872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21597,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454652,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281660868,"statfs":{"total":289910292480,"available":288420728832,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1321257985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.859466"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20255,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715401+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715401+0000","last_peered":"2024-09-16T04:59:57.715401+0000","last_clean":"2024-09-16T04:59:57.715401+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715401+0000","last_undegraded":"2024-09-16T04:59:57.715401+0000","last_fullsized":"2024-09-16T04:59:57.715401+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":22460,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:05.031401+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:05.031401+0000","last_peered":"2024-09-16T05:00:05.031401+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:05.031401+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1233,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2176}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714366+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714366+0000","last_peered":"2024-09-16T04:59:57.714366+0000","last_clean":"2024-09-16T04:59:57.714366+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714366+0000","last_undegraded":"2024-09-16T04:59:57.714366+0000","last_fullsized":"2024-09-16T04:59:57.714366+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21596,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473676,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93898164,"statfs":{"total":96636764160,"available":96151719936,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":484503214},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:12.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:12.784 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:12.785 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:12.805 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-16T05:00:12.568955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22166,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442172,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673348,"statfs":{"total":289910292480,"available":288433508352,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1340,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003323"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20255,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715401+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715401+0000","last_peered":"2024-09-16T04:59:57.715401+0000","last_clean":"2024-09-16T04:59:57.715401+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715401+0000","last_undegraded":"2024-09-16T04:59:57.715401+0000","last_fullsized":"2024-09-16T04:59:57.715401+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":23598,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:10.034558+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:10.034558+0000","last_peered":"2024-09-16T05:00:10.034558+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:10.034558+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1802,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1607}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714366+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714366+0000","last_peered":"2024-09-16T04:59:57.714366+0000","last_clean":"2024-09-16T04:59:57.714366+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714366+0000","last_undegraded":"2024-09-16T04:59:57.714366+0000","last_fullsized":"2024-09-16T04:59:57.714366+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22165,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461196,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910644,"statfs":{"total":96636764160,"available":96164499456,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:12.806 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:13.117 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:13.118 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:13.135 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-16T05:00:12.568955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22166,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442172,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673348,"statfs":{"total":289910292480,"available":288433508352,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1340,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003323"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20255,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715401+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715401+0000","last_peered":"2024-09-16T04:59:57.715401+0000","last_clean":"2024-09-16T04:59:57.715401+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715401+0000","last_undegraded":"2024-09-16T04:59:57.715401+0000","last_fullsized":"2024-09-16T04:59:57.715401+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":23598,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:10.034558+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:10.034558+0000","last_peered":"2024-09-16T05:00:10.034558+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:10.034558+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1802,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1607}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714366+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714366+0000","last_peered":"2024-09-16T04:59:57.714366+0000","last_clean":"2024-09-16T04:59:57.714366+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714366+0000","last_undegraded":"2024-09-16T04:59:57.714366+0000","last_fullsized":"2024-09-16T04:59:57.714366+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22165,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461196,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910644,"statfs":{"total":96636764160,"available":96164499456,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:13.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:00:13.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20021, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.718009+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:57.718009+0000', 'last_peered': '2024-09-16T04:59:57.718009+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:57.718009+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:13.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:00:13.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20470, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.715414+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:57.715414+0000', 'last_peered': '2024-09-16T04:59:57.715414+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:57.715414+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:13.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:00:13.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20032, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.715741+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:57.715741+0000', 'last_peered': '2024-09-16T04:59:57.715741+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:57.715741+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:13.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:00:13.137 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20333, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.714952+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:57.714952+0000', 'last_peered': '2024-09-16T04:59:57.714952+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:57.714952+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:13.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T05:00:13.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20372, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.718401+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:57.718401+0000', 'last_peered': '2024-09-16T04:59:57.718401+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:57.718401+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:13.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T05:00:13.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 23598, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:10.034558+0000', 'last_change': '2024-09-16T04:59:56.595445+0000', 'last_active': '2024-09-16T05:00:10.034558+0000', 'last_peered': '2024-09-16T05:00:10.034558+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T05:00:10.034558+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1608, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1802, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1607}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:13.138 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:00:13.499 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:13.499 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18577/18 objects degraded (103205.556%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":761,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1476784128,"bytes_avail":288433508352,"bytes_total":289910292480,"degraded_objects":18577,"degraded_total":18,"degraded_ratio":1032.0555555555557,"recovering_objects_per_sec":111,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1340,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:13.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:13.519 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:13.817 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:13.818 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:13.836 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-16T05:00:12.568955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22166,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1442172,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281673348,"statfs":{"total":289910292480,"available":288433508352,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1308478465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1340,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1340,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003323"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20255,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715401+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715401+0000","last_peered":"2024-09-16T04:59:57.715401+0000","last_clean":"2024-09-16T04:59:57.715401+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715401+0000","last_undegraded":"2024-09-16T04:59:57.715401+0000","last_fullsized":"2024-09-16T04:59:57.715401+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":23598,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:10.034558+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:10.034558+0000","last_peered":"2024-09-16T05:00:10.034558+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:10.034558+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1608,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1802,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1607}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714366+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714366+0000","last_peered":"2024-09-16T04:59:57.714366+0000","last_clean":"2024-09-16T04:59:57.714366+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714366+0000","last_undegraded":"2024-09-16T04:59:57.714366+0000","last_fullsized":"2024-09-16T04:59:57.714366+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22165,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461196,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93910644,"statfs":{"total":96636764160,"available":96164499456,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":471723694},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:16.839 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:17.180 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:17.181 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:17.199 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-16T05:00:16.570208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22593,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446332,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281669188,"statfs":{"total":289910292480,"available":288429248512,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1767,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003110"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20255,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715401+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715401+0000","last_peered":"2024-09-16T04:59:57.715401+0000","last_clean":"2024-09-16T04:59:57.715401+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715401+0000","last_undegraded":"2024-09-16T04:59:57.715401+0000","last_fullsized":"2024-09-16T04:59:57.715401+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":24452,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:15.039807+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:15.039807+0000","last_peered":"2024-09-16T05:00:15.039807+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:15.039807+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2229,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1180}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714479+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714479+0000","last_peered":"2024-09-16T04:59:57.714479+0000","last_clean":"2024-09-16T04:59:57.714479+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714479+0000","last_undegraded":"2024-09-16T04:59:57.714479+0000","last_fullsized":"2024-09-16T04:59:57.714479+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22592,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593856,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465356,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906484,"statfs":{"total":96636764160,"available":96160239616,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:17.201 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:17.533 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:17.533 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:17.551 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-16T05:00:16.570208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22593,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446332,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281669188,"statfs":{"total":289910292480,"available":288429248512,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1767,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003110"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20255,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715401+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715401+0000","last_peered":"2024-09-16T04:59:57.715401+0000","last_clean":"2024-09-16T04:59:57.715401+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715401+0000","last_undegraded":"2024-09-16T04:59:57.715401+0000","last_fullsized":"2024-09-16T04:59:57.715401+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":24452,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:15.039807+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:15.039807+0000","last_peered":"2024-09-16T05:00:15.039807+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:15.039807+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2229,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1180}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714479+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714479+0000","last_peered":"2024-09-16T04:59:57.714479+0000","last_clean":"2024-09-16T04:59:57.714479+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714479+0000","last_undegraded":"2024-09-16T04:59:57.714479+0000","last_fullsized":"2024-09-16T04:59:57.714479+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22592,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593856,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465356,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906484,"statfs":{"total":96636764160,"available":96160239616,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:17.552 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:00:17.552 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20021, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.718009+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:57.718009+0000', 'last_peered': '2024-09-16T04:59:57.718009+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:57.718009+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:17.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:00:17.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20470, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.715414+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:57.715414+0000', 'last_peered': '2024-09-16T04:59:57.715414+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:57.715414+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:17.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:00:17.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20032, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.715741+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:57.715741+0000', 'last_peered': '2024-09-16T04:59:57.715741+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:57.715741+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:17.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:00:17.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20333, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.714952+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:57.714952+0000', 'last_peered': '2024-09-16T04:59:57.714952+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:57.714952+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:17.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T05:00:17.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20372, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.718401+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:57.718401+0000', 'last_peered': '2024-09-16T04:59:57.718401+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:57.718401+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:17.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T05:00:17.554 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 24452, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:15.039807+0000', 'last_change': '2024-09-16T04:59:56.595445+0000', 'last_active': '2024-09-16T05:00:15.039807+0000', 'last_peered': '2024-09-16T05:00:15.039807+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T05:00:15.039807+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1181, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2229, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1180}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:17.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:00:17.916 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:17.916 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18150/18 objects degraded (100833.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":765,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1481043968,"bytes_avail":288429248512,"bytes_total":289910292480,"degraded_objects":18150,"degraded_total":18,"degraded_ratio":1008.3333333333335,"recovering_objects_per_sec":147,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1767,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:17.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:17.936 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:18.250 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:18.268 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-16T05:00:16.570208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22593,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446332,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281669188,"statfs":{"total":289910292480,"available":288429248512,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1312738305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1767,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003110"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20255,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715401+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715401+0000","last_peered":"2024-09-16T04:59:57.715401+0000","last_clean":"2024-09-16T04:59:57.715401+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715401+0000","last_undegraded":"2024-09-16T04:59:57.715401+0000","last_fullsized":"2024-09-16T04:59:57.715401+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":24452,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:15.039807+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:15.039807+0000","last_peered":"2024-09-16T05:00:15.039807+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:15.039807+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2229,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1180}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714479+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714479+0000","last_peered":"2024-09-16T04:59:57.714479+0000","last_clean":"2024-09-16T04:59:57.714479+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714479+0000","last_undegraded":"2024-09-16T04:59:57.714479+0000","last_fullsized":"2024-09-16T04:59:57.714479+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22592,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593856,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465356,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93906484,"statfs":{"total":96636764160,"available":96160239616,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":475983534},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:21.270 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:21.598 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:21.617 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-16T05:00:20.571753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17643,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23100,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468540,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646980,"statfs":{"total":289910292480,"available":288406507520,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1503,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003880"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20256,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715466+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715466+0000","last_peered":"2024-09-16T04:59:57.715466+0000","last_clean":"2024-09-16T04:59:57.715466+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715466+0000","last_undegraded":"2024-09-16T04:59:57.715466+0000","last_fullsized":"2024-09-16T04:59:57.715466+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":25466,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:20.036253+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:20.036253+0000","last_peered":"2024-09-16T05:00:20.036253+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:20.036253+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":674,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2736,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-673}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714479+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714479+0000","last_peered":"2024-09-16T04:59:57.714479+0000","last_clean":"2024-09-16T04:59:57.714479+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714479+0000","last_undegraded":"2024-09-16T04:59:57.714479+0000","last_fullsized":"2024-09-16T04:59:57.714479+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17643,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23099,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593857,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487564,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884276,"statfs":{"total":96636764160,"available":96137498624,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:21.618 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:21.925 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:21.925 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:21.944 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-16T05:00:20.571753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17643,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23100,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468540,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646980,"statfs":{"total":289910292480,"available":288406507520,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1503,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1503,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003880"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20256,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715466+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715466+0000","last_peered":"2024-09-16T04:59:57.715466+0000","last_clean":"2024-09-16T04:59:57.715466+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715466+0000","last_undegraded":"2024-09-16T04:59:57.715466+0000","last_fullsized":"2024-09-16T04:59:57.715466+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":25466,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:20.036253+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:20.036253+0000","last_peered":"2024-09-16T05:00:20.036253+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:20.036253+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":674,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2736,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-673}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714479+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714479+0000","last_peered":"2024-09-16T04:59:57.714479+0000","last_clean":"2024-09-16T04:59:57.714479+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714479+0000","last_undegraded":"2024-09-16T04:59:57.714479+0000","last_fullsized":"2024-09-16T04:59:57.714479+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17643,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23099,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593857,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487564,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884276,"statfs":{"total":96636764160,"available":96137498624,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:21.945 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:00:21.945 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20021, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.718009+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:57.718009+0000', 'last_peered': '2024-09-16T04:59:57.718009+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:57.718009+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:21.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:00:21.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20470, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.715414+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:57.715414+0000', 'last_peered': '2024-09-16T04:59:57.715414+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:57.715414+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:21.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:00:21.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20032, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.715741+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:57.715741+0000', 'last_peered': '2024-09-16T04:59:57.715741+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:57.715741+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:21.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:00:21.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20333, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.714952+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:57.714952+0000', 'last_peered': '2024-09-16T04:59:57.714952+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:57.714952+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:21.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T05:00:21.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20372, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.718401+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:57.718401+0000', 'last_peered': '2024-09-16T04:59:57.718401+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:57.718401+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:21.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T05:00:21.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 25466, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:20.036253+0000', 'last_change': '2024-09-16T04:59:56.595445+0000', 'last_active': '2024-09-16T05:00:20.036253+0000', 'last_peered': '2024-09-16T05:00:20.036253+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T05:00:20.036253+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 674, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2736, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -673}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:21.947 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:00:22.319 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:22.319 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17643/18 objects degraded (98016.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":770,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1503784960,"bytes_avail":288406507520,"bytes_total":289910292480,"degraded_objects":17643,"degraded_total":18,"degraded_ratio":980.16666666666652,"recovering_objects_per_sec":125,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1503,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:22.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:22.339 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:22.693 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:22.711 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-09-16T05:00:22.572437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17643,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23100,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468540,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646980,"statfs":{"total":289910292480,"available":288406507520,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-934,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":934,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004217"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20256,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715466+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715466+0000","last_peered":"2024-09-16T04:59:57.715466+0000","last_clean":"2024-09-16T04:59:57.715466+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715466+0000","last_undegraded":"2024-09-16T04:59:57.715466+0000","last_fullsized":"2024-09-16T04:59:57.715466+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":25466,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:20.036253+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:20.036253+0000","last_peered":"2024-09-16T05:00:20.036253+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:20.036253+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":674,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2736,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-673}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27015,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714450+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714450+0000","last_peered":"2024-09-16T04:59:57.714450+0000","last_clean":"2024-09-16T04:59:57.714450+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714450+0000","last_undegraded":"2024-09-16T04:59:57.714450+0000","last_fullsized":"2024-09-16T04:59:57.714450+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714479+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714479+0000","last_peered":"2024-09-16T04:59:57.714479+0000","last_clean":"2024-09-16T04:59:57.714479+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714479+0000","last_undegraded":"2024-09-16T04:59:57.714479+0000","last_fullsized":"2024-09-16T04:59:57.714479+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17643,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23099,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593857,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487564,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884276,"statfs":{"total":96636764160,"available":96137498624,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:25.713 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:26.067 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:26.068 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:26.089 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-16T05:00:25.665258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23703,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472700,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642820,"statfs":{"total":289910292480,"available":288402247680,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.095686"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20256,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715466+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715466+0000","last_peered":"2024-09-16T04:59:57.715466+0000","last_clean":"2024-09-16T04:59:57.715466+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715466+0000","last_undegraded":"2024-09-16T04:59:57.715466+0000","last_fullsized":"2024-09-16T04:59:57.715466+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":26672,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:25.041079+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:25.041079+0000","last_peered":"2024-09-16T05:00:25.041079+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:25.041079+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":71,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3339,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-70}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27016,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714507+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714507+0000","last_peered":"2024-09-16T04:59:57.714507+0000","last_clean":"2024-09-16T04:59:57.714507+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714507+0000","last_undegraded":"2024-09-16T04:59:57.714507+0000","last_fullsized":"2024-09-16T04:59:57.714507+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714479+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714479+0000","last_peered":"2024-09-16T04:59:57.714479+0000","last_clean":"2024-09-16T04:59:57.714479+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714479+0000","last_undegraded":"2024-09-16T04:59:57.714479+0000","last_fullsized":"2024-09-16T04:59:57.714479+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23702,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593858,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491724,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880116,"statfs":{"total":96636764160,"available":96133238784,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:26.090 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:26.399 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:26.419 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-16T05:00:25.665258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23703,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472700,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642820,"statfs":{"total":289910292480,"available":288402247680,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.095686"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20256,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715466+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715466+0000","last_peered":"2024-09-16T04:59:57.715466+0000","last_clean":"2024-09-16T04:59:57.715466+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715466+0000","last_undegraded":"2024-09-16T04:59:57.715466+0000","last_fullsized":"2024-09-16T04:59:57.715466+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":26672,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:25.041079+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:25.041079+0000","last_peered":"2024-09-16T05:00:25.041079+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:25.041079+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":71,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3339,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-70}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27016,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714507+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714507+0000","last_peered":"2024-09-16T04:59:57.714507+0000","last_clean":"2024-09-16T04:59:57.714507+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714507+0000","last_undegraded":"2024-09-16T04:59:57.714507+0000","last_fullsized":"2024-09-16T04:59:57.714507+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714479+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714479+0000","last_peered":"2024-09-16T04:59:57.714479+0000","last_clean":"2024-09-16T04:59:57.714479+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714479+0000","last_undegraded":"2024-09-16T04:59:57.714479+0000","last_fullsized":"2024-09-16T04:59:57.714479+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23702,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593858,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491724,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880116,"statfs":{"total":96636764160,"available":96133238784,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:26.420 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:00:26.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20021, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.718009+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T04:59:57.718009+0000', 'last_peered': '2024-09-16T04:59:57.718009+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T04:59:57.718009+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:26.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:00:26.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20470, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.715414+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T04:59:57.715414+0000', 'last_peered': '2024-09-16T04:59:57.715414+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T04:59:57.715414+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:26.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:00:26.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20032, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.715741+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T04:59:57.715741+0000', 'last_peered': '2024-09-16T04:59:57.715741+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T04:59:57.715741+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:26.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:00:26.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20333, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.714952+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T04:59:57.714952+0000', 'last_peered': '2024-09-16T04:59:57.714952+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T04:59:57.714952+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:26.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T05:00:26.421 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 20372, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T04:59:57.718401+0000', 'last_change': '2024-09-16T04:49:38.241708+0000', 'last_active': '2024-09-16T04:59:57.718401+0000', 'last_peered': '2024-09-16T04:59:57.718401+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T04:59:57.718401+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3467, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3465}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:26.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-16T05:00:26.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "51'19876", 'reported_seq': 26672, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:25.041079+0000', 'last_change': '2024-09-16T04:59:56.595445+0000', 'last_active': '2024-09-16T05:00:25.041079+0000', 'last_peered': '2024-09-16T05:00:25.041079+0000', 'last_clean': '2024-09-16T04:49:35.635153+0000', 'last_became_active': '2024-09-16T04:49:38.172218+0000', 'last_became_peered': '2024-09-16T04:49:38.172218+0000', 'last_unstale': '2024-09-16T05:00:25.041079+0000', 'last_undegraded': '2024-09-16T04:49:37.771507+0000', 'last_fullsized': '2024-09-16T04:49:37.652139+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 42, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 71, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19875, 'num_write_kb': 39749, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3339, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -70}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:26.422 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:00:26.881 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:26.881 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17040/18 objects degraded (94666.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":774,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1508044800,"bytes_avail":288402247680,"bytes_total":289910292480,"degraded_objects":17040,"degraded_total":18,"degraded_ratio":946.66666666666652,"recovering_objects_per_sec":110,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1110,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:26.903 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:26.903 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:27.251 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:27.251 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:27.269 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-09-16T05:00:26.666569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23703,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472700,"kb_used_data":164284,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642820,"statfs":{"total":289910292480,"available":288402247680,"internally_reserved":0,"allocated":168226816,"data_stored":167232991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1110,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.095622"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26530,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.718950+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T04:59:57.718950+0000","last_peered":"2024-09-16T04:59:57.718950+0000","last_clean":"2024-09-16T04:59:57.718950+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T04:59:57.718950+0000","last_undegraded":"2024-09-16T04:59:57.718950+0000","last_fullsized":"2024-09-16T04:59:57.718950+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20134,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713213+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T04:59:57.713213+0000","last_peered":"2024-09-16T04:59:57.713213+0000","last_clean":"2024-09-16T04:59:57.713213+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T04:59:57.713213+0000","last_undegraded":"2024-09-16T04:59:57.713213+0000","last_fullsized":"2024-09-16T04:59:57.713213+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20021,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718009+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T04:59:57.718009+0000","last_peered":"2024-09-16T04:59:57.718009+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T04:59:57.718009+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.712664+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T04:59:57.712664+0000","last_peered":"2024-09-16T04:59:57.712664+0000","last_clean":"2024-09-16T04:59:57.712664+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T04:59:57.712664+0000","last_undegraded":"2024-09-16T04:59:57.712664+0000","last_fullsized":"2024-09-16T04:59:57.712664+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20386,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717842+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T04:59:57.717842+0000","last_peered":"2024-09-16T04:59:57.717842+0000","last_clean":"2024-09-16T04:59:57.717842+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T04:59:57.717842+0000","last_undegraded":"2024-09-16T04:59:57.717842+0000","last_fullsized":"2024-09-16T04:59:57.717842+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20470,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715414+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T04:59:57.715414+0000","last_peered":"2024-09-16T04:59:57.715414+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T04:59:57.715414+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.715741+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T04:59:57.715741+0000","last_peered":"2024-09-16T04:59:57.715741+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T04:59:57.715741+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20256,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715466+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T04:59:57.715466+0000","last_peered":"2024-09-16T04:59:57.715466+0000","last_clean":"2024-09-16T04:59:57.715466+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T04:59:57.715466+0000","last_undegraded":"2024-09-16T04:59:57.715466+0000","last_fullsized":"2024-09-16T04:59:57.715466+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.710135+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T04:59:57.710135+0000","last_peered":"2024-09-16T04:59:57.710135+0000","last_clean":"2024-09-16T04:59:57.710135+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T04:59:57.710135+0000","last_undegraded":"2024-09-16T04:59:57.710135+0000","last_fullsized":"2024-09-16T04:59:57.710135+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20333,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.714952+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T04:59:57.714952+0000","last_peered":"2024-09-16T04:59:57.714952+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T04:59:57.714952+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711888+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T04:59:57.711888+0000","last_peered":"2024-09-16T04:59:57.711888+0000","last_clean":"2024-09-16T04:59:57.711888+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T04:59:57.711888+0000","last_undegraded":"2024-09-16T04:59:57.711888+0000","last_fullsized":"2024-09-16T04:59:57.711888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":20372,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T04:59:57.718401+0000","last_change":"2024-09-16T04:49:38.241708+0000","last_active":"2024-09-16T04:59:57.718401+0000","last_peered":"2024-09-16T04:59:57.718401+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T04:59:57.718401+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3467,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3465}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":122,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.715141+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T04:59:57.715141+0000","last_peered":"2024-09-16T04:59:57.715141+0000","last_clean":"2024-09-16T04:59:57.715141+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T04:59:57.715141+0000","last_undegraded":"2024-09-16T04:59:57.715141+0000","last_fullsized":"2024-09-16T04:59:57.715141+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26848,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711998+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T04:59:57.711998+0000","last_peered":"2024-09-16T04:59:57.711998+0000","last_clean":"2024-09-16T04:59:57.711998+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T04:59:57.711998+0000","last_undegraded":"2024-09-16T04:59:57.711998+0000","last_fullsized":"2024-09-16T04:59:57.711998+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.903819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.713502+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T04:59:57.713502+0000","last_peered":"2024-09-16T04:59:57.713502+0000","last_clean":"2024-09-16T04:59:57.713502+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T04:59:57.713502+0000","last_undegraded":"2024-09-16T04:59:57.713502+0000","last_fullsized":"2024-09-16T04:59:57.713502+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26858,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.711515+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T04:59:57.711515+0000","last_peered":"2024-09-16T04:59:57.711515+0000","last_clean":"2024-09-16T04:59:57.711515+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T04:59:57.711515+0000","last_undegraded":"2024-09-16T04:59:57.711515+0000","last_fullsized":"2024-09-16T04:59:57.711515+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":26672,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:25.041079+0000","last_change":"2024-09-16T04:59:56.595445+0000","last_active":"2024-09-16T05:00:25.041079+0000","last_peered":"2024-09-16T05:00:25.041079+0000","last_clean":"2024-09-16T04:49:35.635153+0000","last_became_active":"2024-09-16T04:49:38.172218+0000","last_became_peered":"2024-09-16T04:49:38.172218+0000","last_unstale":"2024-09-16T05:00:25.041079+0000","last_undegraded":"2024-09-16T04:49:37.771507+0000","last_fullsized":"2024-09-16T04:49:37.652139+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":71,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3339,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-70}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714311+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T04:59:57.714311+0000","last_peered":"2024-09-16T04:59:57.714311+0000","last_clean":"2024-09-16T04:59:57.714311+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T04:59:57.714311+0000","last_undegraded":"2024-09-16T04:59:57.714311+0000","last_fullsized":"2024-09-16T04:59:57.714311+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20662,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714625+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T04:59:57.714625+0000","last_peered":"2024-09-16T04:59:57.714625+0000","last_clean":"2024-09-16T04:59:57.714625+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T04:59:57.714625+0000","last_undegraded":"2024-09-16T04:59:57.714625+0000","last_fullsized":"2024-09-16T04:59:57.714625+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:59:58.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.717647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T04:59:57.717647+0000","last_peered":"2024-09-16T04:59:57.717647+0000","last_clean":"2024-09-16T04:59:57.717647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T04:59:57.717647+0000","last_undegraded":"2024-09-16T04:59:57.717647+0000","last_fullsized":"2024-09-16T04:59:57.717647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27016,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714507+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T04:59:57.714507+0000","last_peered":"2024-09-16T04:59:57.714507+0000","last_clean":"2024-09-16T04:59:57.714507+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T04:59:57.714507+0000","last_undegraded":"2024-09-16T04:59:57.714507+0000","last_fullsized":"2024-09-16T04:59:57.714507+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.714479+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T04:59:57.714479+0000","last_peered":"2024-09-16T04:59:57.714479+0000","last_clean":"2024-09-16T04:59:57.714479+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T04:59:57.714479+0000","last_undegraded":"2024-09-16T04:59:57.714479+0000","last_fullsized":"2024-09-16T04:59:57.714479+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.716592+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T04:59:57.716592+0000","last_peered":"2024-09-16T04:59:57.716592+0000","last_clean":"2024-09-16T04:59:57.716592+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T04:59:57.716592+0000","last_undegraded":"2024-09-16T04:59:57.716592+0000","last_fullsized":"2024-09-16T04:59:57.716592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20577,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T04:59:57.898688+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T04:59:57.898688+0000","last_peered":"2024-09-16T04:59:57.898688+0000","last_clean":"2024-09-16T04:59:57.898688+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T04:59:57.898688+0000","last_undegraded":"2024-09-16T04:59:57.898688+0000","last_fullsized":"2024-09-16T04:59:57.898688+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17040,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23702,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166789120,"data_stored":166747372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493212,"kb_used_data":82500,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878628,"statfs":{"total":96636764160,"available":96131715072,"internally_reserved":0,"allocated":84480000,"data_stored":84159162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593858,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491724,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880116,"statfs":{"total":96636764160,"available":96133238784,"internally_reserved":0,"allocated":520192,"data_stored":168043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487764,"kb_used_data":81276,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884076,"statfs":{"total":96636764160,"available":96137293824,"internally_reserved":0,"allocated":83226624,"data_stored":82905786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:30.271 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:30.609 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:30.610 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:30.630 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-16T05:00:30.574741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24324,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464412,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651108,"statfs":{"total":289910292480,"available":288410734592,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002304"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20390,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.676972+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.676972+0000","last_peered":"2024-09-16T05:00:26.676972+0000","last_clean":"2024-09-16T05:00:26.676972+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.676972+0000","last_undegraded":"2024-09-16T05:00:26.676972+0000","last_fullsized":"2024-09-16T05:00:26.676972+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":21477,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:30.126627+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:30.126627+0000","last_peered":"2024-09-16T05:00:30.126627+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:30.126627+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2917}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24323,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593859,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483412,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888428,"statfs":{"total":96636764160,"available":96141750272,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:30.632 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:31.035 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:31.035 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:31.058 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-16T05:00:30.574741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24324,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464412,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651108,"statfs":{"total":289910292480,"available":288410734592,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002304"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20390,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.676972+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.676972+0000","last_peered":"2024-09-16T05:00:26.676972+0000","last_clean":"2024-09-16T05:00:26.676972+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.676972+0000","last_undegraded":"2024-09-16T05:00:26.676972+0000","last_fullsized":"2024-09-16T05:00:26.676972+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":21477,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:30.126627+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:30.126627+0000","last_peered":"2024-09-16T05:00:30.126627+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:30.126627+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2917}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24323,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593859,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483412,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888428,"statfs":{"total":96636764160,"available":96141750272,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:31.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:00:31.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20025, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.666043+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:26.666043+0000', 'last_peered': '2024-09-16T05:00:26.666043+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:26.666043+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:31.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:00:31.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20474, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675099+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:26.675099+0000', 'last_peered': '2024-09-16T05:00:26.675099+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:26.675099+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:31.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:00:31.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20036, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675206+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:26.675206+0000', 'last_peered': '2024-09-16T05:00:26.675206+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:26.675206+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:31.059 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:00:31.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20337, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.665752+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T05:00:26.665752+0000', 'last_peered': '2024-09-16T05:00:26.665752+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:00:26.665752+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:31.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T05:00:31.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 21477, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:30.126627+0000', 'last_change': '2024-09-16T05:00:25.636926+0000', 'last_active': '2024-09-16T05:00:30.126627+0000', 'last_peered': '2024-09-16T05:00:30.126627+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T05:00:30.126627+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2917, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 550, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2917}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:31.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:00:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:31.462 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16419/18 objects degraded (91216.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":779,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1499557888,"bytes_avail":288410734592,"bytes_total":289910292480,"degraded_objects":16419,"degraded_total":18,"degraded_ratio":912.16666666666652,"recovering_objects_per_sec":152,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1224,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:31.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:31.483 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:31.815 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:31.816 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:31.835 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-16T05:00:30.574741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24324,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464412,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651108,"statfs":{"total":289910292480,"available":288410734592,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002304"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20390,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.676972+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.676972+0000","last_peered":"2024-09-16T05:00:26.676972+0000","last_clean":"2024-09-16T05:00:26.676972+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.676972+0000","last_undegraded":"2024-09-16T05:00:26.676972+0000","last_fullsized":"2024-09-16T05:00:26.676972+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":21477,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:30.126627+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:30.126627+0000","last_peered":"2024-09-16T05:00:30.126627+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:30.126627+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2917}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24323,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593859,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483412,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888428,"statfs":{"total":96636764160,"available":96141750272,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:34.837 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:35.661 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:35.679 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-16T05:00:34.575964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24324,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464412,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651108,"statfs":{"total":289910292480,"available":288410734592,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-621,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":621,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.910706"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20390,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.676972+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.676972+0000","last_peered":"2024-09-16T05:00:26.676972+0000","last_clean":"2024-09-16T05:00:26.676972+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.676972+0000","last_undegraded":"2024-09-16T05:00:26.676972+0000","last_fullsized":"2024-09-16T05:00:26.676972+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":21477,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:30.126627+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:30.126627+0000","last_peered":"2024-09-16T05:00:30.126627+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:30.126627+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2917}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24323,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593859,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483412,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888428,"statfs":{"total":96636764160,"available":96141750272,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:35.681 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:35.983 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:35.983 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:36.002 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-16T05:00:34.575964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24324,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464412,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651108,"statfs":{"total":289910292480,"available":288410734592,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-621,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":621,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.910706"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20390,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.676972+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.676972+0000","last_peered":"2024-09-16T05:00:26.676972+0000","last_clean":"2024-09-16T05:00:26.676972+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.676972+0000","last_undegraded":"2024-09-16T05:00:26.676972+0000","last_fullsized":"2024-09-16T05:00:26.676972+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":21477,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:30.126627+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:30.126627+0000","last_peered":"2024-09-16T05:00:30.126627+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:30.126627+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2917,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2917}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16419,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24323,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593859,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483412,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888428,"statfs":{"total":96636764160,"available":96141750272,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:36.004 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:00:36.004 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20025, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.666043+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:26.666043+0000', 'last_peered': '2024-09-16T05:00:26.666043+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:26.666043+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:36.004 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:00:36.004 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20474, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675099+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:26.675099+0000', 'last_peered': '2024-09-16T05:00:26.675099+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:26.675099+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:36.004 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:00:36.004 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20036, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675206+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:26.675206+0000', 'last_peered': '2024-09-16T05:00:26.675206+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:26.675206+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:36.004 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:00:36.004 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20337, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.665752+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T05:00:26.665752+0000', 'last_peered': '2024-09-16T05:00:26.665752+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:00:26.665752+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:36.004 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T05:00:36.004 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 21477, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:30.126627+0000', 'last_change': '2024-09-16T05:00:25.636926+0000', 'last_active': '2024-09-16T05:00:30.126627+0000', 'last_peered': '2024-09-16T05:00:30.126627+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T05:00:30.126627+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2917, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 550, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2917}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:36.005 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:00:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:36.467 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16419/18 objects degraded (91216.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":784,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1499557888,"bytes_avail":288410734592,"bytes_total":289910292480,"degraded_objects":16419,"degraded_total":18,"degraded_ratio":912.16666666666652,"recovering_objects_per_sec":69,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":621,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:36.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:36.486 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:36.814 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:36.832 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-09-16T05:00:36.576353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25013,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468572,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646948,"statfs":{"total":289910292480,"available":288406474752,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002656"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20390,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.676972+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.676972+0000","last_peered":"2024-09-16T05:00:26.676972+0000","last_clean":"2024-09-16T05:00:26.676972+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.676972+0000","last_undegraded":"2024-09-16T05:00:26.676972+0000","last_fullsized":"2024-09-16T05:00:26.676972+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":22855,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:35.134451+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:35.134451+0000","last_peered":"2024-09-16T05:00:35.134451+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:35.134451+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2228}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25012,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593860,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:39.835 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:40.201 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:40.219 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-16T05:00:38.576842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25013,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468572,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646948,"statfs":{"total":289910292480,"available":288406474752,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.910272"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20390,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.676972+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.676972+0000","last_peered":"2024-09-16T05:00:26.676972+0000","last_clean":"2024-09-16T05:00:26.676972+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.676972+0000","last_undegraded":"2024-09-16T05:00:26.676972+0000","last_fullsized":"2024-09-16T05:00:26.676972+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":22855,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:35.134451+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:35.134451+0000","last_peered":"2024-09-16T05:00:35.134451+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:35.134451+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2228}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25012,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593860,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:40.221 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:40.558 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:40.577 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-16T05:00:38.576842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25013,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468572,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646948,"statfs":{"total":289910292480,"available":288406474752,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.910272"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20390,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.676972+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.676972+0000","last_peered":"2024-09-16T05:00:26.676972+0000","last_clean":"2024-09-16T05:00:26.676972+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.676972+0000","last_undegraded":"2024-09-16T05:00:26.676972+0000","last_fullsized":"2024-09-16T05:00:26.676972+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":22855,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:35.134451+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:35.134451+0000","last_peered":"2024-09-16T05:00:35.134451+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:35.134451+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2228}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25012,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593860,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:40.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:00:40.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20025, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.666043+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:26.666043+0000', 'last_peered': '2024-09-16T05:00:26.666043+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:26.666043+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:40.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:00:40.579 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20474, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675099+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:26.675099+0000', 'last_peered': '2024-09-16T05:00:26.675099+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:26.675099+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:40.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:00:40.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20036, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675206+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:26.675206+0000', 'last_peered': '2024-09-16T05:00:26.675206+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:26.675206+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:40.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:00:40.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20337, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.665752+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T05:00:26.665752+0000', 'last_peered': '2024-09-16T05:00:26.665752+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:00:26.665752+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:40.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T05:00:40.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 22855, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:35.134451+0000', 'last_change': '2024-09-16T05:00:25.636926+0000', 'last_active': '2024-09-16T05:00:35.134451+0000', 'last_peered': '2024-09-16T05:00:35.134451+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T05:00:35.134451+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2228, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1239, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2228}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:40.581 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:00:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:40.949 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15358/18 objects degraded (85322.222%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":788,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1508077568,"bytes_avail":288402214912,"bytes_total":289910292480,"degraded_objects":15358,"degraded_total":18,"degraded_ratio":853.22222222222217,"recovering_objects_per_sec":140,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1682,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:40.970 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:40.970 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:41.279 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:41.298 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-09-16T05:00:40.577400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25385,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472732,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642788,"statfs":{"total":289910292480,"available":288402214912,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1682,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1682,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003263"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20391,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.677026+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.677026+0000","last_peered":"2024-09-16T05:00:26.677026+0000","last_clean":"2024-09-16T05:00:26.677026+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.677026+0000","last_undegraded":"2024-09-16T05:00:26.677026+0000","last_fullsized":"2024-09-16T05:00:26.677026+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":23599,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:40.131287+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:40.131287+0000","last_peered":"2024-09-16T05:00:40.131287+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:40.131287+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1856,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1611,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1856}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25384,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593861,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491732,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880108,"statfs":{"total":96636764160,"available":96133230592,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:44.300 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:44.645 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:44.646 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:44.664 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-09-16T05:00:44.578412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25385,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472732,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642788,"statfs":{"total":289910292480,"available":288402214912,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1061,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003104"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20391,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.677026+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.677026+0000","last_peered":"2024-09-16T05:00:26.677026+0000","last_clean":"2024-09-16T05:00:26.677026+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.677026+0000","last_undegraded":"2024-09-16T05:00:26.677026+0000","last_fullsized":"2024-09-16T05:00:26.677026+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":23599,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:40.131287+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:40.131287+0000","last_peered":"2024-09-16T05:00:40.131287+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:40.131287+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1856,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1611,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1856}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25384,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593861,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491732,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880108,"statfs":{"total":96636764160,"available":96133230592,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:44.665 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:44.975 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:44.976 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:44.998 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-09-16T05:00:44.578412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25385,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472732,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642788,"statfs":{"total":289910292480,"available":288402214912,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1061,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003104"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20391,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.677026+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.677026+0000","last_peered":"2024-09-16T05:00:26.677026+0000","last_clean":"2024-09-16T05:00:26.677026+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.677026+0000","last_undegraded":"2024-09-16T05:00:26.677026+0000","last_fullsized":"2024-09-16T05:00:26.677026+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":23599,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:40.131287+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:40.131287+0000","last_peered":"2024-09-16T05:00:40.131287+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:40.131287+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1856,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1611,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1856}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25384,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593861,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491732,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880108,"statfs":{"total":96636764160,"available":96133230592,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:44.999 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:00:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20025, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.666043+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:26.666043+0000', 'last_peered': '2024-09-16T05:00:26.666043+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:26.666043+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:00:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20474, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675099+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:26.675099+0000', 'last_peered': '2024-09-16T05:00:26.675099+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:26.675099+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:00:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20036, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675206+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:26.675206+0000', 'last_peered': '2024-09-16T05:00:26.675206+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:26.675206+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:00:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20337, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.665752+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T05:00:26.665752+0000', 'last_peered': '2024-09-16T05:00:26.665752+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:00:26.665752+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T05:00:45.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 23599, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:40.131287+0000', 'last_change': '2024-09-16T05:00:25.636926+0000', 'last_active': '2024-09-16T05:00:40.131287+0000', 'last_peered': '2024-09-16T05:00:40.131287+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T05:00:40.131287+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1856, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1611, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1856}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:45.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:00:45.374 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:45.374 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15358/18 objects degraded (85322.222%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":793,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1508077568,"bytes_avail":288402214912,"bytes_total":289910292480,"degraded_objects":15358,"degraded_total":18,"degraded_ratio":853.22222222222217,"recovering_objects_per_sec":88,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1061,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:45.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:45.394 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:45.733 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:45.753 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-09-16T05:00:44.578412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25385,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472732,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642788,"statfs":{"total":289910292480,"available":288402214912,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1061,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003104"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20391,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.677026+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.677026+0000","last_peered":"2024-09-16T05:00:26.677026+0000","last_clean":"2024-09-16T05:00:26.677026+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.677026+0000","last_undegraded":"2024-09-16T05:00:26.677026+0000","last_fullsized":"2024-09-16T05:00:26.677026+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":23599,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:40.131287+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:40.131287+0000","last_peered":"2024-09-16T05:00:40.131287+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:40.131287+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1856,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1611,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1856}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25384,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593861,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491732,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880108,"statfs":{"total":96636764160,"available":96133230592,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:48.755 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:49.078 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:49.079 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:49.097 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-09-16T05:00:48.579538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25829,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472732,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642788,"statfs":{"total":289910292480,"available":288402214912,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003184"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20391,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.677026+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.677026+0000","last_peered":"2024-09-16T05:00:26.677026+0000","last_clean":"2024-09-16T05:00:26.677026+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.677026+0000","last_undegraded":"2024-09-16T05:00:26.677026+0000","last_fullsized":"2024-09-16T05:00:26.677026+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":24487,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:45.134371+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:45.134371+0000","last_peered":"2024-09-16T05:00:45.134371+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:45.134371+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2055,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1412}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25828,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491732,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880108,"statfs":{"total":96636764160,"available":96133230592,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:49.098 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:49.428 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:49.429 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:49.447 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-09-16T05:00:48.579538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25829,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472732,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642788,"statfs":{"total":289910292480,"available":288402214912,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003184"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20391,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.677026+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.677026+0000","last_peered":"2024-09-16T05:00:26.677026+0000","last_clean":"2024-09-16T05:00:26.677026+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.677026+0000","last_undegraded":"2024-09-16T05:00:26.677026+0000","last_fullsized":"2024-09-16T05:00:26.677026+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":24487,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:45.134371+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:45.134371+0000","last_peered":"2024-09-16T05:00:45.134371+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:45.134371+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2055,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1412}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25828,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491732,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880108,"statfs":{"total":96636764160,"available":96133230592,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:49.448 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:00:49.448 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20025, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.666043+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:26.666043+0000', 'last_peered': '2024-09-16T05:00:26.666043+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:26.666043+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:49.448 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:00:49.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20474, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675099+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:26.675099+0000', 'last_peered': '2024-09-16T05:00:26.675099+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:26.675099+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:49.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:00:49.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20036, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675206+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:26.675206+0000', 'last_peered': '2024-09-16T05:00:26.675206+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:26.675206+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:49.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:00:49.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20337, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.665752+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T05:00:26.665752+0000', 'last_peered': '2024-09-16T05:00:26.665752+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:00:26.665752+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:49.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T05:00:49.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 24487, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:45.134371+0000', 'last_change': '2024-09-16T05:00:25.636926+0000', 'last_active': '2024-09-16T05:00:45.134371+0000', 'last_peered': '2024-09-16T05:00:45.134371+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T05:00:45.134371+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1412, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2055, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1412}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:49.450 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:00:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:49.831 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14914/18 objects degraded (82855.556%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":797,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1508077568,"bytes_avail":288402214912,"bytes_total":289910292480,"degraded_objects":14914,"degraded_total":18,"degraded_ratio":828.55555555555543,"recovering_objects_per_sec":67,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":816,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:49.851 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:49.851 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:50.226 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:50.226 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:50.245 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-09-16T05:00:48.579538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25829,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472732,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642788,"statfs":{"total":289910292480,"available":288402214912,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-816,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003184"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20391,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.677026+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.677026+0000","last_peered":"2024-09-16T05:00:26.677026+0000","last_clean":"2024-09-16T05:00:26.677026+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.677026+0000","last_undegraded":"2024-09-16T05:00:26.677026+0000","last_fullsized":"2024-09-16T05:00:26.677026+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":24487,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:45.134371+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:45.134371+0000","last_peered":"2024-09-16T05:00:45.134371+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:45.134371+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2055,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1412}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26862,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665693+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665693+0000","last_peered":"2024-09-16T05:00:26.665693+0000","last_clean":"2024-09-16T05:00:26.665693+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665693+0000","last_undegraded":"2024-09-16T05:00:26.665693+0000","last_fullsized":"2024-09-16T05:00:26.665693+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25828,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491732,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880108,"statfs":{"total":96636764160,"available":96133230592,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:53.247 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:53.578 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:53.579 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:53.597 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-16T05:00:52.580842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26314,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460252,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1295859,"kb_avail":281655268,"statfs":{"total":289910292480,"available":288414994432,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1326959617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":929,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003441"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20391,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.677026+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.677026+0000","last_peered":"2024-09-16T05:00:26.677026+0000","last_clean":"2024-09-16T05:00:26.677026+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.677026+0000","last_undegraded":"2024-09-16T05:00:26.677026+0000","last_fullsized":"2024-09-16T05:00:26.677026+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":25457,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:50.137667+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:50.137667+0000","last_peered":"2024-09-16T05:00:50.137667+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:50.137667+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":927,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2540,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26863,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665749+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665749+0000","last_peered":"2024-09-16T05:00:26.665749+0000","last_clean":"2024-09-16T05:00:26.665749+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665749+0000","last_undegraded":"2024-09-16T05:00:26.665749+0000","last_fullsized":"2024-09-16T05:00:26.665749+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26313,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593863,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479252,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892588,"statfs":{"total":96636764160,"available":96146010112,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:53.599 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:53.899 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:53.899 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:53.919 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-16T05:00:52.580842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26314,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460252,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1295859,"kb_avail":281655268,"statfs":{"total":289910292480,"available":288414994432,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1326959617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":929,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003441"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20391,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.677026+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.677026+0000","last_peered":"2024-09-16T05:00:26.677026+0000","last_clean":"2024-09-16T05:00:26.677026+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.677026+0000","last_undegraded":"2024-09-16T05:00:26.677026+0000","last_fullsized":"2024-09-16T05:00:26.677026+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":25457,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:50.137667+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:50.137667+0000","last_peered":"2024-09-16T05:00:50.137667+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:50.137667+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":927,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2540,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26863,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665749+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665749+0000","last_peered":"2024-09-16T05:00:26.665749+0000","last_clean":"2024-09-16T05:00:26.665749+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665749+0000","last_undegraded":"2024-09-16T05:00:26.665749+0000","last_fullsized":"2024-09-16T05:00:26.665749+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26313,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593863,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479252,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892588,"statfs":{"total":96636764160,"available":96146010112,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:53.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:00:53.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20025, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.666043+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:26.666043+0000', 'last_peered': '2024-09-16T05:00:26.666043+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:26.666043+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:53.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:00:53.920 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20474, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675099+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:26.675099+0000', 'last_peered': '2024-09-16T05:00:26.675099+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:26.675099+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:53.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:00:53.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20036, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675206+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:26.675206+0000', 'last_peered': '2024-09-16T05:00:26.675206+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:26.675206+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:53.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:00:53.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20337, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.665752+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T05:00:26.665752+0000', 'last_peered': '2024-09-16T05:00:26.665752+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:00:26.665752+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:53.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T05:00:53.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 25457, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:50.137667+0000', 'last_change': '2024-09-16T05:00:25.636926+0000', 'last_active': '2024-09-16T05:00:50.137667+0000', 'last_peered': '2024-09-16T05:00:50.137667+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T05:00:50.137667+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 927, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2540, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -927}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:53.922 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:00:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:54.308 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14429/18 objects degraded (80161.111%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":802,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1495298048,"bytes_avail":288414994432,"bytes_total":289910292480,"degraded_objects":14429,"degraded_total":18,"degraded_ratio":801.61111111111109,"recovering_objects_per_sec":77,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":929,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:54.328 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:54.329 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:54.664 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:54.664 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:54.682 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-16T05:00:54.581528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26314,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460252,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1295859,"kb_avail":281655268,"statfs":{"total":289910292480,"available":288414994432,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1326959617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":929,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003665"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20391,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.677026+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.677026+0000","last_peered":"2024-09-16T05:00:26.677026+0000","last_clean":"2024-09-16T05:00:26.677026+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.677026+0000","last_undegraded":"2024-09-16T05:00:26.677026+0000","last_fullsized":"2024-09-16T05:00:26.677026+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":25457,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:50.137667+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:50.137667+0000","last_peered":"2024-09-16T05:00:50.137667+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:50.137667+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":927,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2540,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675004+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675004+0000","last_peered":"2024-09-16T05:00:26.675004+0000","last_clean":"2024-09-16T05:00:26.675004+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675004+0000","last_undegraded":"2024-09-16T05:00:26.675004+0000","last_fullsized":"2024-09-16T05:00:26.675004+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26863,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665749+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665749+0000","last_peered":"2024-09-16T05:00:26.665749+0000","last_clean":"2024-09-16T05:00:26.665749+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665749+0000","last_undegraded":"2024-09-16T05:00:26.665749+0000","last_fullsized":"2024-09-16T05:00:26.665749+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14429,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26313,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593863,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479252,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892588,"statfs":{"total":96636764160,"available":96146010112,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:57.684 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:58.015 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:58.033 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-16T05:00:56.581986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13868,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26875,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468572,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646948,"statfs":{"total":289910292480,"available":288406474752,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1490,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003573"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20391,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.677026+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.677026+0000","last_peered":"2024-09-16T05:00:26.677026+0000","last_clean":"2024-09-16T05:00:26.677026+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.677026+0000","last_undegraded":"2024-09-16T05:00:26.677026+0000","last_fullsized":"2024-09-16T05:00:26.677026+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":26579,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:55.133186+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:55.133186+0000","last_peered":"2024-09-16T05:00:55.133186+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:55.133186+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3101,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-366}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675047+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675047+0000","last_peered":"2024-09-16T05:00:26.675047+0000","last_clean":"2024-09-16T05:00:26.675047+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675047+0000","last_undegraded":"2024-09-16T05:00:26.675047+0000","last_fullsized":"2024-09-16T05:00:26.675047+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26863,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665749+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665749+0000","last_peered":"2024-09-16T05:00:26.665749+0000","last_clean":"2024-09-16T05:00:26.665749+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665749+0000","last_undegraded":"2024-09-16T05:00:26.665749+0000","last_fullsized":"2024-09-16T05:00:26.665749+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13868,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26874,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:58.035 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:58.338 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:58.338 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:58.357 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-16T05:00:56.581986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13868,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26875,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468572,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646948,"statfs":{"total":289910292480,"available":288406474752,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1490,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003573"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20391,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.677026+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.677026+0000","last_peered":"2024-09-16T05:00:26.677026+0000","last_clean":"2024-09-16T05:00:26.677026+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.677026+0000","last_undegraded":"2024-09-16T05:00:26.677026+0000","last_fullsized":"2024-09-16T05:00:26.677026+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":26579,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:55.133186+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:55.133186+0000","last_peered":"2024-09-16T05:00:55.133186+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:55.133186+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3101,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-366}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675047+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675047+0000","last_peered":"2024-09-16T05:00:26.675047+0000","last_clean":"2024-09-16T05:00:26.675047+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675047+0000","last_undegraded":"2024-09-16T05:00:26.675047+0000","last_fullsized":"2024-09-16T05:00:26.675047+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26863,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665749+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665749+0000","last_peered":"2024-09-16T05:00:26.665749+0000","last_clean":"2024-09-16T05:00:26.665749+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665749+0000","last_undegraded":"2024-09-16T05:00:26.665749+0000","last_fullsized":"2024-09-16T05:00:26.665749+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13868,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26874,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:00:58.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:00:58.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20025, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.666043+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:26.666043+0000', 'last_peered': '2024-09-16T05:00:26.666043+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:26.666043+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:58.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:00:58.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20474, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675099+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:26.675099+0000', 'last_peered': '2024-09-16T05:00:26.675099+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:26.675099+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:58.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:00:58.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20036, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.675206+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:26.675206+0000', 'last_peered': '2024-09-16T05:00:26.675206+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:26.675206+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:58.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:00:58.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20337, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:26.665752+0000', 'last_change': '2024-09-16T04:49:38.234998+0000', 'last_active': '2024-09-16T05:00:26.665752+0000', 'last_peered': '2024-09-16T05:00:26.665752+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:00:26.665752+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:00:58.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-16T05:00:58.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "51'20218", 'reported_seq': 26579, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:55.133186+0000', 'last_change': '2024-09-16T05:00:25.636926+0000', 'last_active': '2024-09-16T05:00:55.133186+0000', 'last_peered': '2024-09-16T05:00:55.133186+0000', 'last_clean': '2024-09-16T04:49:35.646646+0000', 'last_became_active': '2024-09-16T04:49:38.231323+0000', 'last_became_peered': '2024-09-16T04:49:38.231323+0000', 'last_unstale': '2024-09-16T05:00:55.133186+0000', 'last_undegraded': '2024-09-16T04:49:37.755195+0000', 'last_fullsized': '2024-09-16T04:49:37.652858+0000', 'mapping_epoch': 44, 'log_start': "47'10200", 'ondisk_log_start': "47'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3399", 'last_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.640105+0000', 'objects_scrubbed': 3337, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1407, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 366, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16873, 'num_write_kb': 27072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3101, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -366}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:00:58.359 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:00:58.735 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:58.735 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13868/18 objects degraded (77044.444%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":806,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1503817728,"bytes_avail":288406474752,"bytes_total":289910292480,"degraded_objects":13868,"degraded_total":18,"degraded_ratio":770.44444444444457,"recovering_objects_per_sec":87,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1046,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:00:58.756 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:00:58.756 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:00:59.120 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:00:59.120 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:00:59.142 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-16T05:00:58.648789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13868,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26875,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468572,"kb_used_data":164324,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646948,"statfs":{"total":289910292480,"available":288406474752,"internally_reserved":0,"allocated":168267776,"data_stored":167249346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1046,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1046,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.069250"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26534,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.678570+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:26.678570+0000","last_peered":"2024-09-16T05:00:26.678570+0000","last_clean":"2024-09-16T05:00:26.678570+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:26.678570+0000","last_undegraded":"2024-09-16T05:00:26.678570+0000","last_fullsized":"2024-09-16T05:00:26.678570+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20138,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.666371+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:26.666371+0000","last_peered":"2024-09-16T05:00:26.666371+0000","last_clean":"2024-09-16T05:00:26.666371+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:26.666371+0000","last_undegraded":"2024-09-16T05:00:26.666371+0000","last_fullsized":"2024-09-16T05:00:26.666371+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20025,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.666043+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:26.666043+0000","last_peered":"2024-09-16T05:00:26.666043+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:26.666043+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665977+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:26.665977+0000","last_peered":"2024-09-16T05:00:26.665977+0000","last_clean":"2024-09-16T05:00:26.665977+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:26.665977+0000","last_undegraded":"2024-09-16T05:00:26.665977+0000","last_fullsized":"2024-09-16T05:00:26.665977+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20391,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.677026+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:26.677026+0000","last_peered":"2024-09-16T05:00:26.677026+0000","last_clean":"2024-09-16T05:00:26.677026+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:26.677026+0000","last_undegraded":"2024-09-16T05:00:26.677026+0000","last_fullsized":"2024-09-16T05:00:26.677026+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20474,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675099+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:26.675099+0000","last_peered":"2024-09-16T05:00:26.675099+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:26.675099+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.675206+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:26.675206+0000","last_peered":"2024-09-16T05:00:26.675206+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:26.675206+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20260,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.674906+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:26.674906+0000","last_peered":"2024-09-16T05:00:26.674906+0000","last_clean":"2024-09-16T05:00:26.674906+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:26.674906+0000","last_undegraded":"2024-09-16T05:00:26.674906+0000","last_fullsized":"2024-09-16T05:00:26.674906+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667871+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:26.667871+0000","last_peered":"2024-09-16T05:00:26.667871+0000","last_clean":"2024-09-16T05:00:26.667871+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:26.667871+0000","last_undegraded":"2024-09-16T05:00:26.667871+0000","last_fullsized":"2024-09-16T05:00:26.667871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20337,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:26.665752+0000","last_change":"2024-09-16T04:49:38.234998+0000","last_active":"2024-09-16T05:00:26.665752+0000","last_peered":"2024-09-16T05:00:26.665752+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:00:26.665752+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665923+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:26.665923+0000","last_peered":"2024-09-16T05:00:26.665923+0000","last_clean":"2024-09-16T05:00:26.665923+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:26.665923+0000","last_undegraded":"2024-09-16T05:00:26.665923+0000","last_fullsized":"2024-09-16T05:00:26.665923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":26579,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:55.133186+0000","last_change":"2024-09-16T05:00:25.636926+0000","last_active":"2024-09-16T05:00:55.133186+0000","last_peered":"2024-09-16T05:00:55.133186+0000","last_clean":"2024-09-16T04:49:35.646646+0000","last_became_active":"2024-09-16T04:49:38.231323+0000","last_became_peered":"2024-09-16T04:49:38.231323+0000","last_unstale":"2024-09-16T05:00:55.133186+0000","last_undegraded":"2024-09-16T04:49:37.755195+0000","last_fullsized":"2024-09-16T04:49:37.652858+0000","mapping_epoch":44,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3101,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-366}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.675047+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:26.675047+0000","last_peered":"2024-09-16T05:00:26.675047+0000","last_clean":"2024-09-16T05:00:26.675047+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:26.675047+0000","last_undegraded":"2024-09-16T05:00:26.675047+0000","last_fullsized":"2024-09-16T05:00:26.675047+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26852,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665999+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:26.665999+0000","last_peered":"2024-09-16T05:00:26.665999+0000","last_clean":"2024-09-16T05:00:26.665999+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:26.665999+0000","last_undegraded":"2024-09-16T05:00:26.665999+0000","last_fullsized":"2024-09-16T05:00:26.665999+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:26.902731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668143+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:26.668143+0000","last_peered":"2024-09-16T05:00:26.668143+0000","last_clean":"2024-09-16T05:00:26.668143+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:26.668143+0000","last_undegraded":"2024-09-16T05:00:26.668143+0000","last_fullsized":"2024-09-16T05:00:26.668143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26863,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.665749+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:26.665749+0000","last_peered":"2024-09-16T05:00:26.665749+0000","last_clean":"2024-09-16T05:00:26.665749+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:26.665749+0000","last_undegraded":"2024-09-16T05:00:26.665749+0000","last_fullsized":"2024-09-16T05:00:26.665749+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":19996,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.798010+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:26.798010+0000","last_peered":"2024-09-16T05:00:26.798010+0000","last_clean":"2024-09-16T05:00:26.798010+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:26.798010+0000","last_undegraded":"2024-09-16T05:00:26.798010+0000","last_fullsized":"2024-09-16T05:00:26.798010+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:32.640515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668354+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:26.668354+0000","last_peered":"2024-09-16T05:00:26.668354+0000","last_clean":"2024-09-16T05:00:26.668354+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:26.668354+0000","last_undegraded":"2024-09-16T05:00:26.668354+0000","last_fullsized":"2024-09-16T05:00:26.668354+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20666,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668360+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:26.668360+0000","last_peered":"2024-09-16T05:00:26.668360+0000","last_clean":"2024-09-16T05:00:26.668360+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:26.668360+0000","last_undegraded":"2024-09-16T05:00:26.668360+0000","last_fullsized":"2024-09-16T05:00:26.668360+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:30.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.667564+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:26.667564+0000","last_peered":"2024-09-16T05:00:26.667564+0000","last_clean":"2024-09-16T05:00:26.667564+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:26.667564+0000","last_undegraded":"2024-09-16T05:00:26.667564+0000","last_fullsized":"2024-09-16T05:00:26.667564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27020,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668570+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:26.668570+0000","last_peered":"2024-09-16T05:00:26.668570+0000","last_clean":"2024-09-16T05:00:26.668570+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:26.668570+0000","last_undegraded":"2024-09-16T05:00:26.668570+0000","last_fullsized":"2024-09-16T05:00:26.668570+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668260+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:26.668260+0000","last_peered":"2024-09-16T05:00:26.668260+0000","last_clean":"2024-09-16T05:00:26.668260+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:26.668260+0000","last_undegraded":"2024-09-16T05:00:26.668260+0000","last_fullsized":"2024-09-16T05:00:26.668260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668196+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:26.668196+0000","last_peered":"2024-09-16T05:00:26.668196+0000","last_clean":"2024-09-16T05:00:26.668196+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:26.668196+0000","last_undegraded":"2024-09-16T05:00:26.668196+0000","last_fullsized":"2024-09-16T05:00:26.668196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20581,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-16T05:00:26.668243+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:26.668243+0000","last_peered":"2024-09-16T05:00:26.668243+0000","last_clean":"2024-09-16T05:00:26.668243+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:26.668243+0000","last_undegraded":"2024-09-16T05:00:26.668243+0000","last_fullsized":"2024-09-16T05:00:26.668243+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13868,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26874,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166780928,"data_stored":166746138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493224,"kb_used_data":82516,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878616,"statfs":{"total":96636764160,"available":96131702784,"internally_reserved":0,"allocated":84496384,"data_stored":84165025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487572,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884268,"statfs":{"total":96636764160,"available":96137490432,"internally_reserved":0,"allocated":528384,"data_stored":172672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487776,"kb_used_data":81292,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884064,"statfs":{"total":96636764160,"available":96137281536,"internally_reserved":0,"allocated":83243008,"data_stored":82911649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:02.144 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:02.535 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:02.536 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:02.557 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-16T05:01:00.583418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27453,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472760,"kb_used_data":164348,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642760,"statfs":{"total":289910292480,"available":288402186240,"internally_reserved":0,"allocated":168292352,"data_stored":167259759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002575"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20766,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:00.047597+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:00.047597+0000","last_peered":"2024-09-16T05:01:00.047597+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:00.047597+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3111}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662233+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662233+0000","last_peered":"2024-09-16T05:00:59.662233+0000","last_clean":"2024-09-16T05:00:59.662233+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662233+0000","last_undegraded":"2024-09-16T05:00:59.662233+0000","last_fullsized":"2024-09-16T05:00:59.662233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665658+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665658+0000","last_peered":"2024-09-16T05:00:59.665658+0000","last_clean":"2024-09-16T05:00:59.665658+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665658+0000","last_undegraded":"2024-09-16T05:00:59.665658+0000","last_fullsized":"2024-09-16T05:00:59.665658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27452,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491736,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880104,"statfs":{"total":96636764160,"available":96133226496,"internally_reserved":0,"allocated":528384,"data_stored":174355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487784,"kb_used_data":81300,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884056,"statfs":{"total":96636764160,"available":96137273344,"internally_reserved":0,"allocated":83251200,"data_stored":82914566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:02.559 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:02.879 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:02.880 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:02.905 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-16T05:01:02.584122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27453,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472760,"kb_used_data":164348,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642760,"statfs":{"total":289910292480,"available":288402186240,"internally_reserved":0,"allocated":168292352,"data_stored":167259759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002593"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20766,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:00.047597+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:00.047597+0000","last_peered":"2024-09-16T05:01:00.047597+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:00.047597+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3111}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662233+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662233+0000","last_peered":"2024-09-16T05:00:59.662233+0000","last_clean":"2024-09-16T05:00:59.662233+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662233+0000","last_undegraded":"2024-09-16T05:00:59.662233+0000","last_fullsized":"2024-09-16T05:00:59.662233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665658+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665658+0000","last_peered":"2024-09-16T05:00:59.665658+0000","last_clean":"2024-09-16T05:00:59.665658+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665658+0000","last_undegraded":"2024-09-16T05:00:59.665658+0000","last_fullsized":"2024-09-16T05:00:59.665658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27452,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491736,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880104,"statfs":{"total":96636764160,"available":96133226496,"internally_reserved":0,"allocated":528384,"data_stored":174355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487784,"kb_used_data":81300,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884056,"statfs":{"total":96636764160,"available":96137273344,"internally_reserved":0,"allocated":83251200,"data_stored":82914566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:02.907 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:01:02.907 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20029, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.662959+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:59.662959+0000', 'last_peered': '2024-09-16T05:00:59.662959+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:59.662959+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:02.907 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:01:02.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20478, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667571+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:59.667571+0000', 'last_peered': '2024-09-16T05:00:59.667571+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:59.667571+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:02.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:01:02.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20040, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667139+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:59.667139+0000', 'last_peered': '2024-09-16T05:00:59.667139+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:59.667139+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:02.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:01:02.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 20766, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:00.047597+0000', 'last_change': '2024-09-16T05:00:58.242238+0000', 'last_active': '2024-09-16T05:01:00.047597+0000', 'last_peered': '2024-09-16T05:01:00.047597+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:01:00.047597+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 212, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3111}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:02.909 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:01:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:03.282 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13290/18 objects degraded (73833.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":811,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1508106240,"bytes_avail":288402186240,"bytes_total":289910292480,"degraded_objects":13290,"degraded_total":18,"degraded_ratio":738.33333333333348,"recovering_objects_per_sec":142,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1139,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:01:03.302 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:01:03.303 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:03.634 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:03.634 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:03.651 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-16T05:01:02.584122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27453,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472760,"kb_used_data":164348,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642760,"statfs":{"total":289910292480,"available":288402186240,"internally_reserved":0,"allocated":168292352,"data_stored":167259759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002593"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20766,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:00.047597+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:00.047597+0000","last_peered":"2024-09-16T05:01:00.047597+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:00.047597+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3111}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662233+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662233+0000","last_peered":"2024-09-16T05:00:59.662233+0000","last_clean":"2024-09-16T05:00:59.662233+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662233+0000","last_undegraded":"2024-09-16T05:00:59.662233+0000","last_fullsized":"2024-09-16T05:00:59.662233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665658+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665658+0000","last_peered":"2024-09-16T05:00:59.665658+0000","last_clean":"2024-09-16T05:00:59.665658+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665658+0000","last_undegraded":"2024-09-16T05:00:59.665658+0000","last_fullsized":"2024-09-16T05:00:59.665658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27452,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491736,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880104,"statfs":{"total":96636764160,"available":96133226496,"internally_reserved":0,"allocated":528384,"data_stored":174355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487784,"kb_used_data":81300,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884056,"statfs":{"total":96636764160,"available":96137273344,"internally_reserved":0,"allocated":83251200,"data_stored":82914566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:06.654 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:07.013 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:07.013 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:07.033 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-16T05:01:06.585035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28132,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460288,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1295859,"kb_avail":281655232,"statfs":{"total":289910292480,"available":288414957568,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1326959617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002085"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":22124,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:05.052410+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:05.052410+0000","last_peered":"2024-09-16T05:01:05.052410+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:05.052410+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":891,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2432}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662233+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662233+0000","last_peered":"2024-09-16T05:00:59.662233+0000","last_clean":"2024-09-16T05:00:59.662233+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662233+0000","last_undegraded":"2024-09-16T05:00:59.662233+0000","last_fullsized":"2024-09-16T05:00:59.662233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665658+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665658+0000","last_peered":"2024-09-16T05:00:59.665658+0000","last_clean":"2024-09-16T05:00:59.665658+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665658+0000","last_undegraded":"2024-09-16T05:00:59.665658+0000","last_fullsized":"2024-09-16T05:00:59.665658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28131,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479260,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892580,"statfs":{"total":96636764160,"available":96146001920,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:07.035 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:07.391 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:07.392 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:07.411 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-16T05:01:06.585035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28132,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460288,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1295859,"kb_avail":281655232,"statfs":{"total":289910292480,"available":288414957568,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1326959617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002085"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":22124,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:05.052410+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:05.052410+0000","last_peered":"2024-09-16T05:01:05.052410+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:05.052410+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":891,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2432}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662233+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662233+0000","last_peered":"2024-09-16T05:00:59.662233+0000","last_clean":"2024-09-16T05:00:59.662233+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662233+0000","last_undegraded":"2024-09-16T05:00:59.662233+0000","last_fullsized":"2024-09-16T05:00:59.662233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665658+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665658+0000","last_peered":"2024-09-16T05:00:59.665658+0000","last_clean":"2024-09-16T05:00:59.665658+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665658+0000","last_undegraded":"2024-09-16T05:00:59.665658+0000","last_fullsized":"2024-09-16T05:00:59.665658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28131,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479260,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892580,"statfs":{"total":96636764160,"available":96146001920,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:07.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:01:07.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20029, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.662959+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:59.662959+0000', 'last_peered': '2024-09-16T05:00:59.662959+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:59.662959+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:07.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:01:07.412 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20478, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667571+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:59.667571+0000', 'last_peered': '2024-09-16T05:00:59.667571+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:59.667571+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:07.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:01:07.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20040, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667139+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:59.667139+0000', 'last_peered': '2024-09-16T05:00:59.667139+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:59.667139+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:07.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:01:07.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 22124, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:05.052410+0000', 'last_change': '2024-09-16T05:00:58.242238+0000', 'last_active': '2024-09-16T05:01:05.052410+0000', 'last_peered': '2024-09-16T05:01:05.052410+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:01:05.052410+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2432, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 891, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2432}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:07.413 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:01:07.834 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:07.834 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12611/18 objects degraded (70061.111%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":815,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1495334912,"bytes_avail":288414957568,"bytes_total":289910292480,"degraded_objects":12611,"degraded_total":18,"degraded_ratio":700.61111111111109,"recovering_objects_per_sec":157,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1257,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:01:07.855 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:01:07.855 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:08.167 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:08.167 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:08.185 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-16T05:01:06.585035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28132,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460288,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1295859,"kb_avail":281655232,"statfs":{"total":289910292480,"available":288414957568,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1326959617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002085"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":22124,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:05.052410+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:05.052410+0000","last_peered":"2024-09-16T05:01:05.052410+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:05.052410+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":891,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2432}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662233+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662233+0000","last_peered":"2024-09-16T05:00:59.662233+0000","last_clean":"2024-09-16T05:00:59.662233+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662233+0000","last_undegraded":"2024-09-16T05:00:59.662233+0000","last_fullsized":"2024-09-16T05:00:59.662233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665658+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665658+0000","last_peered":"2024-09-16T05:00:59.665658+0000","last_clean":"2024-09-16T05:00:59.665658+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665658+0000","last_undegraded":"2024-09-16T05:00:59.665658+0000","last_fullsized":"2024-09-16T05:00:59.665658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28131,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479260,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892580,"statfs":{"total":96636764160,"available":96146001920,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:11.188 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:11.553 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:11.553 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:11.573 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-16T05:01:10.586679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28718,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464448,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651072,"statfs":{"total":289910292480,"available":288410697728,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1843,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1843,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.926201"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":23296,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:10.049492+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:10.049492+0000","last_peered":"2024-09-16T05:01:10.049492+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:10.049492+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1477,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1846}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662233+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662233+0000","last_peered":"2024-09-16T05:00:59.662233+0000","last_clean":"2024-09-16T05:00:59.662233+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662233+0000","last_undegraded":"2024-09-16T05:00:59.662233+0000","last_fullsized":"2024-09-16T05:00:59.662233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665658+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665658+0000","last_peered":"2024-09-16T05:00:59.665658+0000","last_clean":"2024-09-16T05:00:59.665658+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665658+0000","last_undegraded":"2024-09-16T05:00:59.665658+0000","last_fullsized":"2024-09-16T05:00:59.665658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28717,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483420,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888420,"statfs":{"total":96636764160,"available":96141742080,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:11.574 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:11.878 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:11.897 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-16T05:01:10.586679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28718,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464448,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651072,"statfs":{"total":289910292480,"available":288410697728,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1843,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1843,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.926201"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":23296,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:10.049492+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:10.049492+0000","last_peered":"2024-09-16T05:01:10.049492+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:10.049492+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1477,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1846}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662233+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662233+0000","last_peered":"2024-09-16T05:00:59.662233+0000","last_clean":"2024-09-16T05:00:59.662233+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662233+0000","last_undegraded":"2024-09-16T05:00:59.662233+0000","last_fullsized":"2024-09-16T05:00:59.662233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665658+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665658+0000","last_peered":"2024-09-16T05:00:59.665658+0000","last_clean":"2024-09-16T05:00:59.665658+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665658+0000","last_undegraded":"2024-09-16T05:00:59.665658+0000","last_fullsized":"2024-09-16T05:00:59.665658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28717,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483420,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888420,"statfs":{"total":96636764160,"available":96141742080,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:11.898 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:01:11.898 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20029, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.662959+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:59.662959+0000', 'last_peered': '2024-09-16T05:00:59.662959+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:59.662959+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:11.898 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:01:11.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20478, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667571+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:59.667571+0000', 'last_peered': '2024-09-16T05:00:59.667571+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:59.667571+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:11.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:01:11.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20040, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667139+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:59.667139+0000', 'last_peered': '2024-09-16T05:00:59.667139+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:59.667139+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:11.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:01:11.899 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 23296, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:10.049492+0000', 'last_change': '2024-09-16T05:00:58.242238+0000', 'last_active': '2024-09-16T05:01:10.049492+0000', 'last_peered': '2024-09-16T05:01:10.049492+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:01:10.049492+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1846, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1477, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1846}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:11.899 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:01:12.281 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:12.281 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12025/18 objects degraded (66805.556%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":820,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1499594752,"bytes_avail":288410697728,"bytes_total":289910292480,"degraded_objects":12025,"degraded_total":18,"degraded_ratio":668.05555555555543,"recovering_objects_per_sec":168,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1843,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:01:12.300 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:01:12.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:12.612 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:12.613 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:12.631 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-09-16T05:01:12.587379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28718,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464448,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651072,"statfs":{"total":289910292480,"available":288410697728,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1265,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003960"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":23296,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:10.049492+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:10.049492+0000","last_peered":"2024-09-16T05:01:10.049492+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:10.049492+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1477,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1846}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662233+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662233+0000","last_peered":"2024-09-16T05:00:59.662233+0000","last_clean":"2024-09-16T05:00:59.662233+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662233+0000","last_undegraded":"2024-09-16T05:00:59.662233+0000","last_fullsized":"2024-09-16T05:00:59.662233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665658+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665658+0000","last_peered":"2024-09-16T05:00:59.665658+0000","last_clean":"2024-09-16T05:00:59.665658+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665658+0000","last_undegraded":"2024-09-16T05:00:59.665658+0000","last_fullsized":"2024-09-16T05:00:59.665658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28717,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483420,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888420,"statfs":{"total":96636764160,"available":96141742080,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:15.636 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:15.983 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:15.984 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:16.004 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-16T05:01:14.587940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28718,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464448,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651072,"statfs":{"total":289910292480,"available":288410697728,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1265,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003817"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":23296,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:10.049492+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:10.049492+0000","last_peered":"2024-09-16T05:01:10.049492+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:10.049492+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1477,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1846}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662233+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662233+0000","last_peered":"2024-09-16T05:00:59.662233+0000","last_clean":"2024-09-16T05:00:59.662233+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662233+0000","last_undegraded":"2024-09-16T05:00:59.662233+0000","last_fullsized":"2024-09-16T05:00:59.662233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665658+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665658+0000","last_peered":"2024-09-16T05:00:59.665658+0000","last_clean":"2024-09-16T05:00:59.665658+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665658+0000","last_undegraded":"2024-09-16T05:00:59.665658+0000","last_fullsized":"2024-09-16T05:00:59.665658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28717,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483420,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888420,"statfs":{"total":96636764160,"available":96141742080,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:16.005 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:16.339 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:16.340 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:16.358 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-16T05:01:14.587940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28718,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464448,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651072,"statfs":{"total":289910292480,"available":288410697728,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1265,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003817"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":23296,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:10.049492+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:10.049492+0000","last_peered":"2024-09-16T05:01:10.049492+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:10.049492+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1846,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1477,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1846}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662233+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662233+0000","last_peered":"2024-09-16T05:00:59.662233+0000","last_clean":"2024-09-16T05:00:59.662233+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662233+0000","last_undegraded":"2024-09-16T05:00:59.662233+0000","last_fullsized":"2024-09-16T05:00:59.662233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665658+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665658+0000","last_peered":"2024-09-16T05:00:59.665658+0000","last_clean":"2024-09-16T05:00:59.665658+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665658+0000","last_undegraded":"2024-09-16T05:00:59.665658+0000","last_fullsized":"2024-09-16T05:00:59.665658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28717,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483420,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888420,"statfs":{"total":96636764160,"available":96141742080,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:16.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:01:16.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20029, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.662959+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:59.662959+0000', 'last_peered': '2024-09-16T05:00:59.662959+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:59.662959+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:16.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:01:16.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20478, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667571+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:59.667571+0000', 'last_peered': '2024-09-16T05:00:59.667571+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:59.667571+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:16.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:01:16.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20040, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667139+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:59.667139+0000', 'last_peered': '2024-09-16T05:00:59.667139+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:59.667139+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:16.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:01:16.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 23296, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:10.049492+0000', 'last_change': '2024-09-16T05:00:58.242238+0000', 'last_active': '2024-09-16T05:01:10.049492+0000', 'last_peered': '2024-09-16T05:01:10.049492+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:01:10.049492+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1846, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1477, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1846}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:16.361 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:01:16.747 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:16.747 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11618/18 objects degraded (64544.444%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":824,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1503854592,"bytes_avail":288406437888,"bytes_total":289910292480,"degraded_objects":11618,"degraded_total":18,"degraded_ratio":645.44444444444446,"recovering_objects_per_sec":139,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1672,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:01:16.766 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:01:16.767 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:17.097 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:17.097 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:17.116 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2024-09-16T05:01:16.588683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11618,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29125,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468608,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646912,"statfs":{"total":289910292480,"available":288406437888,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1672,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004111"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":24110,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:15.054459+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:15.054459+0000","last_peered":"2024-09-16T05:01:15.054459+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:15.054459+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1439,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1439}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662345+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662345+0000","last_peered":"2024-09-16T05:00:59.662345+0000","last_clean":"2024-09-16T05:00:59.662345+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662345+0000","last_undegraded":"2024-09-16T05:00:59.662345+0000","last_fullsized":"2024-09-16T05:00:59.662345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665658+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665658+0000","last_peered":"2024-09-16T05:00:59.665658+0000","last_clean":"2024-09-16T05:00:59.665658+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665658+0000","last_undegraded":"2024-09-16T05:00:59.665658+0000","last_fullsized":"2024-09-16T05:00:59.665658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11618,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29124,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487580,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884260,"statfs":{"total":96636764160,"available":96137482240,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:20.118 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:20.447 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:20.447 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:20.466 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-16T05:01:18.589205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11618,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29125,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468608,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646912,"statfs":{"total":289910292480,"available":288406437888,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":993,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004170"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":24110,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:15.054459+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:15.054459+0000","last_peered":"2024-09-16T05:01:15.054459+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:15.054459+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1439,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1884,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1439}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662345+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662345+0000","last_peered":"2024-09-16T05:00:59.662345+0000","last_clean":"2024-09-16T05:00:59.662345+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662345+0000","last_undegraded":"2024-09-16T05:00:59.662345+0000","last_fullsized":"2024-09-16T05:00:59.662345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665658+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665658+0000","last_peered":"2024-09-16T05:00:59.665658+0000","last_clean":"2024-09-16T05:00:59.665658+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665658+0000","last_undegraded":"2024-09-16T05:00:59.665658+0000","last_fullsized":"2024-09-16T05:00:59.665658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11618,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29124,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487580,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884260,"statfs":{"total":96636764160,"available":96137482240,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:20.467 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:20.781 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:20.782 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:20.800 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-16T05:01:20.589742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29575,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472768,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642752,"statfs":{"total":289910292480,"available":288402178048,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1443,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1443,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003785"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":25010,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:20.058442+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:20.058442+0000","last_peered":"2024-09-16T05:01:20.058442+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:20.058442+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-989}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662345+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662345+0000","last_peered":"2024-09-16T05:00:59.662345+0000","last_clean":"2024-09-16T05:00:59.662345+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662345+0000","last_undegraded":"2024-09-16T05:00:59.662345+0000","last_fullsized":"2024-09-16T05:00:59.662345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665724+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665724+0000","last_peered":"2024-09-16T05:00:59.665724+0000","last_clean":"2024-09-16T05:00:59.665724+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665724+0000","last_undegraded":"2024-09-16T05:00:59.665724+0000","last_fullsized":"2024-09-16T05:00:59.665724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29574,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491740,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880100,"statfs":{"total":96636764160,"available":96133222400,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:20.801 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:01:20.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20029, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.662959+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:59.662959+0000', 'last_peered': '2024-09-16T05:00:59.662959+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:59.662959+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:20.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:01:20.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20478, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667571+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:59.667571+0000', 'last_peered': '2024-09-16T05:00:59.667571+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:59.667571+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:20.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:01:20.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20040, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667139+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:59.667139+0000', 'last_peered': '2024-09-16T05:00:59.667139+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:59.667139+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:20.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:01:20.802 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 25010, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:20.058442+0000', 'last_change': '2024-09-16T05:00:58.242238+0000', 'last_active': '2024-09-16T05:01:20.058442+0000', 'last_peered': '2024-09-16T05:01:20.058442+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:01:20.058442+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 989, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2334, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -989}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:20.803 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:01:21.178 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:21.178 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11168/18 objects degraded (62044.444%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":828,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1508114432,"bytes_avail":288402178048,"bytes_total":289910292480,"degraded_objects":11168,"degraded_total":18,"degraded_ratio":620.44444444444446,"recovering_objects_per_sec":120,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1443,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:01:21.198 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:01:21.198 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:21.513 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:21.514 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:21.531 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-16T05:01:20.589742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29575,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472768,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642752,"statfs":{"total":289910292480,"available":288402178048,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1443,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1443,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003785"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":25010,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:20.058442+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:20.058442+0000","last_peered":"2024-09-16T05:01:20.058442+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:20.058442+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-989}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662345+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662345+0000","last_peered":"2024-09-16T05:00:59.662345+0000","last_clean":"2024-09-16T05:00:59.662345+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662345+0000","last_undegraded":"2024-09-16T05:00:59.662345+0000","last_fullsized":"2024-09-16T05:00:59.662345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665724+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665724+0000","last_peered":"2024-09-16T05:00:59.665724+0000","last_clean":"2024-09-16T05:00:59.665724+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665724+0000","last_undegraded":"2024-09-16T05:00:59.665724+0000","last_fullsized":"2024-09-16T05:00:59.665724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29574,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491740,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880100,"statfs":{"total":96636764160,"available":96133222400,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:24.534 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:24.885 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:24.886 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:24.905 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-16T05:01:24.590900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29575,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472768,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642752,"statfs":{"total":289910292480,"available":288402178048,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-857,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":857,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003521"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":25010,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:20.058442+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:20.058442+0000","last_peered":"2024-09-16T05:01:20.058442+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:20.058442+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-989}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662345+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662345+0000","last_peered":"2024-09-16T05:00:59.662345+0000","last_clean":"2024-09-16T05:00:59.662345+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662345+0000","last_undegraded":"2024-09-16T05:00:59.662345+0000","last_fullsized":"2024-09-16T05:00:59.662345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665724+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665724+0000","last_peered":"2024-09-16T05:00:59.665724+0000","last_clean":"2024-09-16T05:00:59.665724+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665724+0000","last_undegraded":"2024-09-16T05:00:59.665724+0000","last_fullsized":"2024-09-16T05:00:59.665724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29574,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491740,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880100,"statfs":{"total":96636764160,"available":96133222400,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:24.906 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:25.223 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:25.224 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:25.242 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-16T05:01:24.590900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29575,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472768,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642752,"statfs":{"total":289910292480,"available":288402178048,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-857,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":857,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003521"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":25010,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:20.058442+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:20.058442+0000","last_peered":"2024-09-16T05:01:20.058442+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:20.058442+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-989}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662345+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662345+0000","last_peered":"2024-09-16T05:00:59.662345+0000","last_clean":"2024-09-16T05:00:59.662345+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662345+0000","last_undegraded":"2024-09-16T05:00:59.662345+0000","last_fullsized":"2024-09-16T05:00:59.662345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665724+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665724+0000","last_peered":"2024-09-16T05:00:59.665724+0000","last_clean":"2024-09-16T05:00:59.665724+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665724+0000","last_undegraded":"2024-09-16T05:00:59.665724+0000","last_fullsized":"2024-09-16T05:00:59.665724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29574,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491740,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880100,"statfs":{"total":96636764160,"available":96133222400,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:25.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:01:25.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20029, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.662959+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:59.662959+0000', 'last_peered': '2024-09-16T05:00:59.662959+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:59.662959+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:25.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:01:25.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20478, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667571+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:59.667571+0000', 'last_peered': '2024-09-16T05:00:59.667571+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:59.667571+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:25.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:01:25.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20040, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667139+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:59.667139+0000', 'last_peered': '2024-09-16T05:00:59.667139+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:59.667139+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:25.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:01:25.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 25010, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:20.058442+0000', 'last_change': '2024-09-16T05:00:58.242238+0000', 'last_active': '2024-09-16T05:01:20.058442+0000', 'last_peered': '2024-09-16T05:01:20.058442+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:01:20.058442+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 989, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2334, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -989}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:25.244 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:01:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:25.612 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11168/18 objects degraded (62044.444%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":833,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1508114432,"bytes_avail":288402178048,"bytes_total":289910292480,"degraded_objects":11168,"degraded_total":18,"degraded_ratio":620.44444444444446,"recovering_objects_per_sec":71,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":857,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:01:25.632 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:01:25.632 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:25.945 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:25.946 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:25.964 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-16T05:01:24.590900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29575,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472768,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642752,"statfs":{"total":289910292480,"available":288402178048,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-857,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":857,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003521"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":25010,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:20.058442+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:20.058442+0000","last_peered":"2024-09-16T05:01:20.058442+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:20.058442+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2334,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-989}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662345+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662345+0000","last_peered":"2024-09-16T05:00:59.662345+0000","last_clean":"2024-09-16T05:00:59.662345+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662345+0000","last_undegraded":"2024-09-16T05:00:59.662345+0000","last_fullsized":"2024-09-16T05:00:59.662345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665724+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665724+0000","last_peered":"2024-09-16T05:00:59.665724+0000","last_clean":"2024-09-16T05:00:59.665724+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665724+0000","last_undegraded":"2024-09-16T05:00:59.665724+0000","last_fullsized":"2024-09-16T05:00:59.665724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29574,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491740,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880100,"statfs":{"total":96636764160,"available":96133222400,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:28.967 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:29.374 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:29.394 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2024-09-16T05:01:28.700045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30126,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460288,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1295859,"kb_avail":281655232,"statfs":{"total":289910292480,"available":288414957568,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1326959617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1001,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1001,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.110839"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":26112,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:25.057671+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:25.057671+0000","last_peered":"2024-09-16T05:01:25.057671+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:25.057671+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2885,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-438}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662345+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662345+0000","last_peered":"2024-09-16T05:00:59.662345+0000","last_clean":"2024-09-16T05:00:59.662345+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662345+0000","last_undegraded":"2024-09-16T05:00:59.662345+0000","last_fullsized":"2024-09-16T05:00:59.662345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665724+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665724+0000","last_peered":"2024-09-16T05:00:59.665724+0000","last_clean":"2024-09-16T05:00:59.665724+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665724+0000","last_undegraded":"2024-09-16T05:00:59.665724+0000","last_fullsized":"2024-09-16T05:00:59.665724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30125,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479260,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892580,"statfs":{"total":96636764160,"available":96146001920,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:29.395 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:29.731 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:29.732 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:29.759 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2024-09-16T05:01:29.698764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30126,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460288,"kb_used_data":164364,"kb_used_omap":12,"kb_used_meta":1295859,"kb_avail":281655232,"statfs":{"total":289910292480,"available":288414957568,"internally_reserved":0,"allocated":168308736,"data_stored":167265551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1326959617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":551,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.109021"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667531+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:00:59.667531+0000","last_peered":"2024-09-16T05:00:59.667531+0000","last_clean":"2024-09-16T05:00:59.667531+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:00:59.667531+0000","last_undegraded":"2024-09-16T05:00:59.667531+0000","last_fullsized":"2024-09-16T05:00:59.667531+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20142,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663361+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:00:59.663361+0000","last_peered":"2024-09-16T05:00:59.663361+0000","last_clean":"2024-09-16T05:00:59.663361+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:00:59.663361+0000","last_undegraded":"2024-09-16T05:00:59.663361+0000","last_fullsized":"2024-09-16T05:00:59.663361+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20029,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.662959+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:00:59.662959+0000","last_peered":"2024-09-16T05:00:59.662959+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:00:59.662959+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20446,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662552+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:00:59.662552+0000","last_peered":"2024-09-16T05:00:59.662552+0000","last_clean":"2024-09-16T05:00:59.662552+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:00:59.662552+0000","last_undegraded":"2024-09-16T05:00:59.662552+0000","last_fullsized":"2024-09-16T05:00:59.662552+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20395,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667542+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:00:59.667542+0000","last_peered":"2024-09-16T05:00:59.667542+0000","last_clean":"2024-09-16T05:00:59.667542+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:00:59.667542+0000","last_undegraded":"2024-09-16T05:00:59.667542+0000","last_fullsized":"2024-09-16T05:00:59.667542+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20478,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667571+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:00:59.667571+0000","last_peered":"2024-09-16T05:00:59.667571+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:00:59.667571+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20040,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:00:59.667139+0000","last_change":"2024-09-16T04:49:38.244751+0000","last_active":"2024-09-16T05:00:59.667139+0000","last_peered":"2024-09-16T05:00:59.667139+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:00:59.667139+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3399}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20264,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.666947+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:00:59.666947+0000","last_peered":"2024-09-16T05:00:59.666947+0000","last_clean":"2024-09-16T05:00:59.666947+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:00:59.666947+0000","last_undegraded":"2024-09-16T05:00:59.666947+0000","last_fullsized":"2024-09-16T05:00:59.666947+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.661878+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:00:59.661878+0000","last_peered":"2024-09-16T05:00:59.661878+0000","last_clean":"2024-09-16T05:00:59.661878+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:00:59.661878+0000","last_undegraded":"2024-09-16T05:00:59.661878+0000","last_fullsized":"2024-09-16T05:00:59.661878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":26112,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:25.057671+0000","last_change":"2024-09-16T05:00:58.242238+0000","last_active":"2024-09-16T05:01:25.057671+0000","last_peered":"2024-09-16T05:01:25.057671+0000","last_clean":"2024-09-16T04:49:35.601987+0000","last_became_active":"2024-09-16T04:49:38.228357+0000","last_became_peered":"2024-09-16T04:49:38.228357+0000","last_unstale":"2024-09-16T05:01:25.057671+0000","last_undegraded":"2024-09-16T04:49:37.789906+0000","last_fullsized":"2024-09-16T04:49:37.646039+0000","mapping_epoch":44,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":438,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2885,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-438}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662345+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:00:59.662345+0000","last_peered":"2024-09-16T05:00:59.662345+0000","last_clean":"2024-09-16T05:00:59.662345+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:00:59.662345+0000","last_undegraded":"2024-09-16T05:00:59.662345+0000","last_fullsized":"2024-09-16T05:00:59.662345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27321,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.817107+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:00:59.817107+0000","last_peered":"2024-09-16T05:00:59.817107+0000","last_clean":"2024-09-16T05:00:59.817107+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:00:59.817107+0000","last_undegraded":"2024-09-16T05:00:59.817107+0000","last_fullsized":"2024-09-16T05:00:59.817107+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":131,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.667437+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:00:59.667437+0000","last_peered":"2024-09-16T05:00:59.667437+0000","last_clean":"2024-09-16T05:00:59.667437+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:00:59.667437+0000","last_undegraded":"2024-09-16T05:00:59.667437+0000","last_fullsized":"2024-09-16T05:00:59.667437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26856,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662483+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:00:59.662483+0000","last_peered":"2024-09-16T05:00:59.662483+0000","last_clean":"2024-09-16T05:00:59.662483+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:00:59.662483+0000","last_undegraded":"2024-09-16T05:00:59.662483+0000","last_fullsized":"2024-09-16T05:00:59.662483+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663411+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:00:59.663411+0000","last_peered":"2024-09-16T05:00:59.663411+0000","last_clean":"2024-09-16T05:00:59.663411+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:00:59.663411+0000","last_undegraded":"2024-09-16T05:00:59.663411+0000","last_fullsized":"2024-09-16T05:00:59.663411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26867,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.662000+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:00:59.662000+0000","last_peered":"2024-09-16T05:00:59.662000+0000","last_clean":"2024-09-16T05:00:59.662000+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:00:59.662000+0000","last_undegraded":"2024-09-16T05:00:59.662000+0000","last_fullsized":"2024-09-16T05:00:59.662000+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20000,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664773+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:00:59.664773+0000","last_peered":"2024-09-16T05:00:59.664773+0000","last_clean":"2024-09-16T05:00:59.664773+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:00:59.664773+0000","last_undegraded":"2024-09-16T05:00:59.664773+0000","last_fullsized":"2024-09-16T05:00:59.664773+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663236+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:00:59.663236+0000","last_peered":"2024-09-16T05:00:59.663236+0000","last_clean":"2024-09-16T05:00:59.663236+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:00:59.663236+0000","last_undegraded":"2024-09-16T05:00:59.663236+0000","last_fullsized":"2024-09-16T05:00:59.663236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20670,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664247+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:00:59.664247+0000","last_peered":"2024-09-16T05:00:59.664247+0000","last_clean":"2024-09-16T05:00:59.664247+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:00:59.664247+0000","last_undegraded":"2024-09-16T05:00:59.664247+0000","last_fullsized":"2024-09-16T05:00:59.664247+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:04.554398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665724+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:00:59.665724+0000","last_peered":"2024-09-16T05:00:59.665724+0000","last_clean":"2024-09-16T05:00:59.665724+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:00:59.665724+0000","last_undegraded":"2024-09-16T05:00:59.665724+0000","last_fullsized":"2024-09-16T05:00:59.665724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27024,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.663717+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:00:59.663717+0000","last_peered":"2024-09-16T05:00:59.663717+0000","last_clean":"2024-09-16T05:00:59.663717+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:00:59.663717+0000","last_undegraded":"2024-09-16T05:00:59.663717+0000","last_fullsized":"2024-09-16T05:00:59.663717+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664025+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:00:59.664025+0000","last_peered":"2024-09-16T05:00:59.664025+0000","last_clean":"2024-09-16T05:00:59.664025+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:00:59.664025+0000","last_undegraded":"2024-09-16T05:00:59.664025+0000","last_fullsized":"2024-09-16T05:00:59.664025+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.664529+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:00:59.664529+0000","last_peered":"2024-09-16T05:00:59.664529+0000","last_clean":"2024-09-16T05:00:59.664529+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:00:59.664529+0000","last_undegraded":"2024-09-16T05:00:59.664529+0000","last_fullsized":"2024-09-16T05:00:59.664529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20585,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T05:00:59.665236+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:00:59.665236+0000","last_peered":"2024-09-16T05:00:59.665236+0000","last_clean":"2024-09-16T05:00:59.665236+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:00:59.665236+0000","last_undegraded":"2024-09-16T05:00:59.665236+0000","last_fullsized":"2024-09-16T05:00:59.665236+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30125,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166772736,"data_stored":166744904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493240,"kb_used_data":82532,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878600,"statfs":{"total":96636764160,"available":96131686400,"internally_reserved":0,"allocated":84512768,"data_stored":84170838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479260,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892580,"statfs":{"total":96636764160,"available":96146001920,"internally_reserved":0,"allocated":536576,"data_stored":177251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487788,"kb_used_data":81308,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884052,"statfs":{"total":96636764160,"available":96137269248,"internally_reserved":0,"allocated":83259392,"data_stored":82917462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:29.760 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:01:29.760 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20029, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.662959+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:00:59.662959+0000', 'last_peered': '2024-09-16T05:00:59.662959+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:00:59.662959+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:29.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:01:29.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20478, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667571+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:00:59.667571+0000', 'last_peered': '2024-09-16T05:00:59.667571+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:00:59.667571+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:29.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:01:29.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20040, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:00:59.667139+0000', 'last_change': '2024-09-16T04:49:38.244751+0000', 'last_active': '2024-09-16T05:00:59.667139+0000', 'last_peered': '2024-09-16T05:00:59.667139+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:00:59.667139+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:29.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-16T05:01:29.761 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "51'20081", 'reported_seq': 26112, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:25.057671+0000', 'last_change': '2024-09-16T05:00:58.242238+0000', 'last_active': '2024-09-16T05:01:25.057671+0000', 'last_peered': '2024-09-16T05:01:25.057671+0000', 'last_clean': '2024-09-16T04:49:35.601987+0000', 'last_became_active': '2024-09-16T04:49:38.228357+0000', 'last_became_peered': '2024-09-16T04:49:38.228357+0000', 'last_unstale': '2024-09-16T05:01:25.057671+0000', 'last_undegraded': '2024-09-16T04:49:37.789906+0000', 'last_fullsized': '2024-09-16T04:49:37.646039+0000', 'mapping_epoch': 44, 'log_start': "47'10000", 'ondisk_log_start': "47'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3289", 'last_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:32.744910+0000', 'objects_scrubbed': 3216, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 438, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16832, 'num_write_kb': 27232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2885, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -438}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:29.761 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:01:30.245 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:30.245 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10617/18 objects degraded (58983.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":838,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1495334912,"bytes_avail":288414957568,"bytes_total":289910292480,"degraded_objects":10617,"degraded_total":18,"degraded_ratio":589.83333333333337,"recovering_objects_per_sec":83,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1001,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:01:30.268 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:01:30.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:30.645 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:30.646 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:30.665 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-09-16T05:01:30.592840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30773,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464472,"kb_used_data":164380,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651048,"statfs":{"total":289910292480,"available":288410673152,"internally_reserved":0,"allocated":168325120,"data_stored":167272993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1198,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002620"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20482,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.709240+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.709240+0000","last_peered":"2024-09-16T05:01:29.709240+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.709240+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20463,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:30.144122+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:30.144122+0000","last_peered":"2024-09-16T05:01:30.144122+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:30.144122+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702333+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702333+0000","last_peered":"2024-09-16T05:01:29.702333+0000","last_clean":"2024-09-16T05:01:29.702333+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702333+0000","last_undegraded":"2024-09-16T05:01:29.702333+0000","last_fullsized":"2024-09-16T05:01:29.702333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20674,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702725+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702725+0000","last_peered":"2024-09-16T05:01:29.702725+0000","last_clean":"2024-09-16T05:01:29.702725+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702725+0000","last_undegraded":"2024-09-16T05:01:29.702725+0000","last_fullsized":"2024-09-16T05:01:29.702725+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.174212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30772,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493248,"kb_used_data":82540,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878592,"statfs":{"total":96636764160,"available":96131678208,"internally_reserved":0,"allocated":84520960,"data_stored":84173730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483424,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888416,"statfs":{"total":96636764160,"available":96141737984,"internally_reserved":0,"allocated":536576,"data_stored":178909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487800,"kb_used_data":81316,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884040,"statfs":{"total":96636764160,"available":96137256960,"internally_reserved":0,"allocated":83267584,"data_stored":82920354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:33.667 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:34.060 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:34.061 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:34.080 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-16T05:01:32.593226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30773,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464472,"kb_used_data":164380,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651048,"statfs":{"total":289910292480,"available":288410673152,"internally_reserved":0,"allocated":168325120,"data_stored":167272993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1198,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002325"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20482,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.709240+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.709240+0000","last_peered":"2024-09-16T05:01:29.709240+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.709240+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20463,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:30.144122+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:30.144122+0000","last_peered":"2024-09-16T05:01:30.144122+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:30.144122+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702333+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702333+0000","last_peered":"2024-09-16T05:01:29.702333+0000","last_clean":"2024-09-16T05:01:29.702333+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702333+0000","last_undegraded":"2024-09-16T05:01:29.702333+0000","last_fullsized":"2024-09-16T05:01:29.702333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20674,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702725+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702725+0000","last_peered":"2024-09-16T05:01:29.702725+0000","last_clean":"2024-09-16T05:01:29.702725+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702725+0000","last_undegraded":"2024-09-16T05:01:29.702725+0000","last_fullsized":"2024-09-16T05:01:29.702725+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.174212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30772,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493248,"kb_used_data":82540,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878592,"statfs":{"total":96636764160,"available":96131678208,"internally_reserved":0,"allocated":84520960,"data_stored":84173730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483424,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888416,"statfs":{"total":96636764160,"available":96141737984,"internally_reserved":0,"allocated":536576,"data_stored":178909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487800,"kb_used_data":81316,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884040,"statfs":{"total":96636764160,"available":96137256960,"internally_reserved":0,"allocated":83267584,"data_stored":82920354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:34.081 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:34.446 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:34.447 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:34.470 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-16T05:01:32.593226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30773,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464472,"kb_used_data":164380,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651048,"statfs":{"total":289910292480,"available":288410673152,"internally_reserved":0,"allocated":168325120,"data_stored":167272993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1198,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002325"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20482,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.709240+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.709240+0000","last_peered":"2024-09-16T05:01:29.709240+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.709240+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20463,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:30.144122+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:30.144122+0000","last_peered":"2024-09-16T05:01:30.144122+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:30.144122+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702333+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702333+0000","last_peered":"2024-09-16T05:01:29.702333+0000","last_clean":"2024-09-16T05:01:29.702333+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702333+0000","last_undegraded":"2024-09-16T05:01:29.702333+0000","last_fullsized":"2024-09-16T05:01:29.702333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20674,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702725+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702725+0000","last_peered":"2024-09-16T05:01:29.702725+0000","last_clean":"2024-09-16T05:01:29.702725+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702725+0000","last_undegraded":"2024-09-16T05:01:29.702725+0000","last_fullsized":"2024-09-16T05:01:29.702725+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.174212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30772,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493248,"kb_used_data":82540,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878592,"statfs":{"total":96636764160,"available":96131678208,"internally_reserved":0,"allocated":84520960,"data_stored":84173730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483424,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888416,"statfs":{"total":96636764160,"available":96141737984,"internally_reserved":0,"allocated":536576,"data_stored":178909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487800,"kb_used_data":81316,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884040,"statfs":{"total":96636764160,"available":96137256960,"internally_reserved":0,"allocated":83267584,"data_stored":82920354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:34.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:01:34.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20033, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:29.701202+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:01:29.701202+0000', 'last_peered': '2024-09-16T05:01:29.701202+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:01:29.701202+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:34.471 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:01:34.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20482, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:29.709240+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:29.709240+0000', 'last_peered': '2024-09-16T05:01:29.709240+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:29.709240+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:34.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:01:34.472 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 20463, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:30.144122+0000', 'last_change': '2024-09-16T05:01:28.328858+0000', 'last_active': '2024-09-16T05:01:30.144122+0000', 'last_peered': '2024-09-16T05:01:30.144122+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:01:30.144122+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3192, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 209, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:34.472 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:01:34.900 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:34.900 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9970/18 objects degraded (55388.889%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":842,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1499619328,"bytes_avail":288410673152,"bytes_total":289910292480,"degraded_objects":9970,"degraded_total":18,"degraded_ratio":553.88888888888891,"recovering_objects_per_sec":80,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":647,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:01:34.921 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:01:34.921 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:35.277 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:35.278 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:35.300 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-09-16T05:01:34.593749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30773,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464472,"kb_used_data":164380,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651048,"statfs":{"total":289910292480,"available":288410673152,"internally_reserved":0,"allocated":168325120,"data_stored":167272993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002351"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20482,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.709240+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.709240+0000","last_peered":"2024-09-16T05:01:29.709240+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.709240+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20463,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:30.144122+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:30.144122+0000","last_peered":"2024-09-16T05:01:30.144122+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:30.144122+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702333+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702333+0000","last_peered":"2024-09-16T05:01:29.702333+0000","last_clean":"2024-09-16T05:01:29.702333+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702333+0000","last_undegraded":"2024-09-16T05:01:29.702333+0000","last_fullsized":"2024-09-16T05:01:29.702333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20674,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702725+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702725+0000","last_peered":"2024-09-16T05:01:29.702725+0000","last_clean":"2024-09-16T05:01:29.702725+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702725+0000","last_undegraded":"2024-09-16T05:01:29.702725+0000","last_fullsized":"2024-09-16T05:01:29.702725+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.174212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30772,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493248,"kb_used_data":82540,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878592,"statfs":{"total":96636764160,"available":96131678208,"internally_reserved":0,"allocated":84520960,"data_stored":84173730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483424,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888416,"statfs":{"total":96636764160,"available":96141737984,"internally_reserved":0,"allocated":536576,"data_stored":178909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487800,"kb_used_data":81316,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884040,"statfs":{"total":96636764160,"available":96137256960,"internally_reserved":0,"allocated":83267584,"data_stored":82920354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:38.303 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:38.679 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:38.680 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:38.703 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-16T05:01:38.594837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31452,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468656,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646864,"statfs":{"total":289910292480,"available":288406388736,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.894791"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":21822,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:35.145824+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:35.145824+0000","last_peered":"2024-09-16T05:01:35.145824+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:35.145824+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":888,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702333+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702333+0000","last_peered":"2024-09-16T05:01:29.702333+0000","last_clean":"2024-09-16T05:01:29.702333+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702333+0000","last_undegraded":"2024-09-16T05:01:29.702333+0000","last_fullsized":"2024-09-16T05:01:29.702333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31451,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:38.704 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:39.031 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:39.032 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:39.050 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-16T05:01:38.594837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31452,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468656,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646864,"statfs":{"total":289910292480,"available":288406388736,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.894791"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":21822,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:35.145824+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:35.145824+0000","last_peered":"2024-09-16T05:01:35.145824+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:35.145824+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":888,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702333+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702333+0000","last_peered":"2024-09-16T05:01:29.702333+0000","last_clean":"2024-09-16T05:01:29.702333+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702333+0000","last_undegraded":"2024-09-16T05:01:29.702333+0000","last_fullsized":"2024-09-16T05:01:29.702333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31451,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:39.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:01:39.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20033, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:29.701202+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:01:29.701202+0000', 'last_peered': '2024-09-16T05:01:29.701202+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:01:29.701202+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:39.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:01:39.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20483, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:29.710327+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:29.710327+0000', 'last_peered': '2024-09-16T05:01:29.710327+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:29.710327+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:39.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:01:39.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 21822, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:35.145824+0000', 'last_change': '2024-09-16T05:01:28.328858+0000', 'last_active': '2024-09-16T05:01:35.145824+0000', 'last_peered': '2024-09-16T05:01:35.145824+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:01:35.145824+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2513, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 888, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2513}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:39.052 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:01:39.426 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:39.426 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9291/18 objects degraded (51616.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":847,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1503903744,"bytes_avail":288406388736,"bytes_total":289910292480,"degraded_objects":9291,"degraded_total":18,"degraded_ratio":516.16666666666663,"recovering_objects_per_sec":134,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1326,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:01:39.447 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:01:39.447 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:39.798 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:39.816 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-16T05:01:38.594837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31452,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468656,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646864,"statfs":{"total":289910292480,"available":288406388736,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1326,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.894791"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":21822,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:35.145824+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:35.145824+0000","last_peered":"2024-09-16T05:01:35.145824+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:35.145824+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":888,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702333+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702333+0000","last_peered":"2024-09-16T05:01:29.702333+0000","last_clean":"2024-09-16T05:01:29.702333+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702333+0000","last_undegraded":"2024-09-16T05:01:29.702333+0000","last_fullsized":"2024-09-16T05:01:29.702333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31451,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:42.819 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:43.160 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:43.182 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-16T05:01:42.596012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32111,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460336,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1295859,"kb_avail":281655184,"statfs":{"total":289910292480,"available":288414908416,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1326959617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1338,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003172"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":23140,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:40.140278+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:40.140278+0000","last_peered":"2024-09-16T05:01:40.140278+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:40.140278+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1547,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702404+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702404+0000","last_peered":"2024-09-16T05:01:29.702404+0000","last_clean":"2024-09-16T05:01:29.702404+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702404+0000","last_undegraded":"2024-09-16T05:01:29.702404+0000","last_fullsized":"2024-09-16T05:01:29.702404+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32110,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479268,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892572,"statfs":{"total":96636764160,"available":96145993728,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:43.184 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:43.505 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:43.505 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:43.525 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-16T05:01:42.596012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32111,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460336,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1295859,"kb_avail":281655184,"statfs":{"total":289910292480,"available":288414908416,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1326959617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1338,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003172"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":23140,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:40.140278+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:40.140278+0000","last_peered":"2024-09-16T05:01:40.140278+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:40.140278+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1547,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702404+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702404+0000","last_peered":"2024-09-16T05:01:29.702404+0000","last_clean":"2024-09-16T05:01:29.702404+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702404+0000","last_undegraded":"2024-09-16T05:01:29.702404+0000","last_fullsized":"2024-09-16T05:01:29.702404+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32110,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479268,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892572,"statfs":{"total":96636764160,"available":96145993728,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:43.526 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:01:43.526 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20033, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:29.701202+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:01:29.701202+0000', 'last_peered': '2024-09-16T05:01:29.701202+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:01:29.701202+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:43.526 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:01:43.526 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20483, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:29.710327+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:29.710327+0000', 'last_peered': '2024-09-16T05:01:29.710327+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:29.710327+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:43.526 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:01:43.526 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 23140, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:40.140278+0000', 'last_change': '2024-09-16T05:01:28.328858+0000', 'last_active': '2024-09-16T05:01:40.140278+0000', 'last_peered': '2024-09-16T05:01:40.140278+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:01:40.140278+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1854, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1547, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1854}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:43.527 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:01:43.906 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:43.906 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8632/18 objects degraded (47955.556%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":851,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1495384064,"bytes_avail":288414908416,"bytes_total":289910292480,"degraded_objects":8632,"degraded_total":18,"degraded_ratio":479.55555555555554,"recovering_objects_per_sec":111,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1338,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:01:43.927 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:01:43.927 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:44.251 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:44.252 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:44.268 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-16T05:01:42.596012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32111,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460336,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1295859,"kb_avail":281655184,"statfs":{"total":289910292480,"available":288414908416,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1326959617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1338,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1338,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003172"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":23140,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:40.140278+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:40.140278+0000","last_peered":"2024-09-16T05:01:40.140278+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:40.140278+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1854,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1547,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702404+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702404+0000","last_peered":"2024-09-16T05:01:29.702404+0000","last_clean":"2024-09-16T05:01:29.702404+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702404+0000","last_undegraded":"2024-09-16T05:01:29.702404+0000","last_fullsized":"2024-09-16T05:01:29.702404+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32110,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479268,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892572,"statfs":{"total":96636764160,"available":96145993728,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:47.271 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:47.611 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:47.612 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:47.630 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-16T05:01:46.597224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32517,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464496,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651024,"statfs":{"total":289910292480,"available":288410648576,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1744,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1744,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003474"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":23952,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:45.140585+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:45.140585+0000","last_peered":"2024-09-16T05:01:45.140585+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:45.140585+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1448}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702404+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702404+0000","last_peered":"2024-09-16T05:01:29.702404+0000","last_clean":"2024-09-16T05:01:29.702404+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702404+0000","last_undegraded":"2024-09-16T05:01:29.702404+0000","last_fullsized":"2024-09-16T05:01:29.702404+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32516,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593874,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483428,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888412,"statfs":{"total":96636764160,"available":96141733888,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:47.632 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:47.940 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:47.940 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:47.959 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-16T05:01:46.597224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32517,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464496,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651024,"statfs":{"total":289910292480,"available":288410648576,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1744,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1744,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003474"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":23952,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:45.140585+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:45.140585+0000","last_peered":"2024-09-16T05:01:45.140585+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:45.140585+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1448}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702404+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702404+0000","last_peered":"2024-09-16T05:01:29.702404+0000","last_clean":"2024-09-16T05:01:29.702404+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702404+0000","last_undegraded":"2024-09-16T05:01:29.702404+0000","last_fullsized":"2024-09-16T05:01:29.702404+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32516,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593874,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483428,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888412,"statfs":{"total":96636764160,"available":96141733888,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:47.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:01:47.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20033, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:29.701202+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:01:29.701202+0000', 'last_peered': '2024-09-16T05:01:29.701202+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:01:29.701202+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:47.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:01:47.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20483, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:29.710327+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:29.710327+0000', 'last_peered': '2024-09-16T05:01:29.710327+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:29.710327+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:47.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:01:47.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 23952, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:45.140585+0000', 'last_change': '2024-09-16T05:01:28.328858+0000', 'last_active': '2024-09-16T05:01:45.140585+0000', 'last_peered': '2024-09-16T05:01:45.140585+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:01:45.140585+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1448, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1953, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1448}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:47.961 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:01:48.354 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:48.354 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8226/18 objects degraded (45700.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":856,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1499643904,"bytes_avail":288410648576,"bytes_total":289910292480,"degraded_objects":8226,"degraded_total":18,"degraded_ratio":457,"recovering_objects_per_sec":145,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1744,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:01:48.378 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:01:48.379 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:48.707 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:48.726 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2024-09-16T05:01:48.597887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32517,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464496,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281651024,"statfs":{"total":289910292480,"available":288410648576,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1065,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003773"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":23952,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:45.140585+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:45.140585+0000","last_peered":"2024-09-16T05:01:45.140585+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:45.140585+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1448}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702404+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702404+0000","last_peered":"2024-09-16T05:01:29.702404+0000","last_clean":"2024-09-16T05:01:29.702404+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702404+0000","last_undegraded":"2024-09-16T05:01:29.702404+0000","last_fullsized":"2024-09-16T05:01:29.702404+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32516,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593874,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483428,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888412,"statfs":{"total":96636764160,"available":96141733888,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:51.728 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:52.050 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:52.051 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:52.074 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-16T05:01:50.598375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7776,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32967,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468656,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646864,"statfs":{"total":289910292480,"available":288406388736,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1515,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003538"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":24852,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:50.147032+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:50.147032+0000","last_peered":"2024-09-16T05:01:50.147032+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:50.147032+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2403,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702404+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702404+0000","last_peered":"2024-09-16T05:01:29.702404+0000","last_clean":"2024-09-16T05:01:29.702404+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702404+0000","last_undegraded":"2024-09-16T05:01:29.702404+0000","last_fullsized":"2024-09-16T05:01:29.702404+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7776,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32966,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593875,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:52.075 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:52.386 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:52.403 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-16T05:01:50.598375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7776,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32967,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468656,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646864,"statfs":{"total":289910292480,"available":288406388736,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1515,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003538"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":24852,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:50.147032+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:50.147032+0000","last_peered":"2024-09-16T05:01:50.147032+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:50.147032+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2403,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702404+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702404+0000","last_peered":"2024-09-16T05:01:29.702404+0000","last_clean":"2024-09-16T05:01:29.702404+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702404+0000","last_undegraded":"2024-09-16T05:01:29.702404+0000","last_fullsized":"2024-09-16T05:01:29.702404+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7776,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32966,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593875,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:52.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:01:52.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20033, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:29.701202+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:01:29.701202+0000', 'last_peered': '2024-09-16T05:01:29.701202+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:01:29.701202+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:52.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:01:52.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20483, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:29.710327+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:29.710327+0000', 'last_peered': '2024-09-16T05:01:29.710327+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:29.710327+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:52.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:01:52.405 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 24852, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:50.147032+0000', 'last_change': '2024-09-16T05:01:28.328858+0000', 'last_active': '2024-09-16T05:01:50.147032+0000', 'last_peered': '2024-09-16T05:01:50.147032+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:01:50.147032+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 998, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2403, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -998}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:52.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:01:52.768 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:52.768 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7776/18 objects degraded (43200.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":860,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1503903744,"bytes_avail":288406388736,"bytes_total":289910292480,"degraded_objects":7776,"degraded_total":18,"degraded_ratio":432,"recovering_objects_per_sec":71,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":856,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:01:52.793 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:01:52.793 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:53.095 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:53.095 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:53.113 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-16T05:01:52.599154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7776,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32967,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468656,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646864,"statfs":{"total":289910292480,"available":288406388736,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-856,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":856,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003887"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":24852,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:50.147032+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:50.147032+0000","last_peered":"2024-09-16T05:01:50.147032+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:50.147032+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2403,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702404+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702404+0000","last_peered":"2024-09-16T05:01:29.702404+0000","last_clean":"2024-09-16T05:01:29.702404+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702404+0000","last_undegraded":"2024-09-16T05:01:29.702404+0000","last_fullsized":"2024-09-16T05:01:29.702404+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7776,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32966,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593875,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:56.116 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:56.466 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:56.466 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:56.486 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-16T05:01:54.599489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7776,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32967,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468656,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646864,"statfs":{"total":289910292480,"available":288406388736,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-856,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":856,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003476"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711096+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711096+0000","last_peered":"2024-09-16T05:01:29.711096+0000","last_clean":"2024-09-16T05:01:29.711096+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711096+0000","last_undegraded":"2024-09-16T05:01:29.711096+0000","last_fullsized":"2024-09-16T05:01:29.711096+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":24852,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:50.147032+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:50.147032+0000","last_peered":"2024-09-16T05:01:50.147032+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:50.147032+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2403,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-998}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702404+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702404+0000","last_peered":"2024-09-16T05:01:29.702404+0000","last_clean":"2024-09-16T05:01:29.702404+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702404+0000","last_undegraded":"2024-09-16T05:01:29.702404+0000","last_fullsized":"2024-09-16T05:01:29.702404+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26860,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700687+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700687+0000","last_peered":"2024-09-16T05:01:29.700687+0000","last_clean":"2024-09-16T05:01:29.700687+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700687+0000","last_undegraded":"2024-09-16T05:01:29.700687+0000","last_fullsized":"2024-09-16T05:01:29.700687+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:32.117756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7776,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32966,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593875,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:56.487 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:56.794 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:56.795 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:56.813 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2024-09-16T05:01:56.600059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33499,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468656,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646864,"statfs":{"total":289910292480,"available":288406388736,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1388,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003443"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26543,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711122+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711122+0000","last_peered":"2024-09-16T05:01:29.711122+0000","last_clean":"2024-09-16T05:01:29.711122+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711122+0000","last_undegraded":"2024-09-16T05:01:29.711122+0000","last_fullsized":"2024-09-16T05:01:29.711122+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":25916,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:55.153214+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:55.153214+0000","last_peered":"2024-09-16T05:01:55.153214+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:55.153214+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2935,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-466}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702404+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702404+0000","last_peered":"2024-09-16T05:01:29.702404+0000","last_clean":"2024-09-16T05:01:29.702404+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702404+0000","last_undegraded":"2024-09-16T05:01:29.702404+0000","last_fullsized":"2024-09-16T05:01:29.702404+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26861,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700743+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700743+0000","last_peered":"2024-09-16T05:01:29.700743+0000","last_clean":"2024-09-16T05:01:29.700743+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700743+0000","last_undegraded":"2024-09-16T05:01:29.700743+0000","last_fullsized":"2024-09-16T05:01:29.700743+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33498,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593876,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:01:56.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:01:56.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20033, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:29.701202+0000', 'last_change': '2024-09-16T04:49:38.237072+0000', 'last_active': '2024-09-16T05:01:29.701202+0000', 'last_peered': '2024-09-16T05:01:29.701202+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:01:29.701202+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3320, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3319}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:01:56.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:01:56.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20483, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:29.710327+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:29.710327+0000', 'last_peered': '2024-09-16T05:01:29.710327+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:29.710327+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:56.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-16T05:01:56.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "51'19860", 'reported_seq': 25916, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:55.153214+0000', 'last_change': '2024-09-16T05:01:28.328858+0000', 'last_active': '2024-09-16T05:01:55.153214+0000', 'last_peered': '2024-09-16T05:01:55.153214+0000', 'last_clean': '2024-09-16T04:49:35.625965+0000', 'last_became_active': '2024-09-16T04:49:38.237533+0000', 'last_became_peered': '2024-09-16T04:49:38.237533+0000', 'last_unstale': '2024-09-16T05:01:55.153214+0000', 'last_undegraded': '2024-09-16T04:49:37.763430+0000', 'last_fullsized': '2024-09-16T04:49:37.646127+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3345", 'last_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.780500+0000', 'objects_scrubbed': 3339, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 579, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 466, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16515, 'num_write_kb': 26352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2935, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -466}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:01:56.816 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:01:57.200 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:57.200 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7244/18 objects degraded (40244.444%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":864,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1503903744,"bytes_avail":288406388736,"bytes_total":289910292480,"degraded_objects":7244,"degraded_total":18,"degraded_ratio":402.44444444444446,"recovering_objects_per_sec":115,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1388,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:01:57.220 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:01:57.220 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:01:57.546 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:01:57.547 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:01:57.565 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2024-09-16T05:01:56.600059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33499,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468656,"kb_used_data":164404,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646864,"statfs":{"total":289910292480,"available":288406388736,"internally_reserved":0,"allocated":168349696,"data_stored":167281606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1388,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003443"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26543,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.711122+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:29.711122+0000","last_peered":"2024-09-16T05:01:29.711122+0000","last_clean":"2024-09-16T05:01:29.711122+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:29.711122+0000","last_undegraded":"2024-09-16T05:01:29.711122+0000","last_fullsized":"2024-09-16T05:01:29.711122+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20146,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.701179+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:29.701179+0000","last_peered":"2024-09-16T05:01:29.701179+0000","last_clean":"2024-09-16T05:01:29.701179+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:29.701179+0000","last_undegraded":"2024-09-16T05:01:29.701179+0000","last_fullsized":"2024-09-16T05:01:29.701179+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.701202+0000","last_change":"2024-09-16T04:49:38.237072+0000","last_active":"2024-09-16T05:01:29.701202+0000","last_peered":"2024-09-16T05:01:29.701202+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:01:29.701202+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20450,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700986+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:29.700986+0000","last_peered":"2024-09-16T05:01:29.700986+0000","last_clean":"2024-09-16T05:01:29.700986+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:29.700986+0000","last_undegraded":"2024-09-16T05:01:29.700986+0000","last_fullsized":"2024-09-16T05:01:29.700986+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20399,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709581+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:29.709581+0000","last_peered":"2024-09-16T05:01:29.709581+0000","last_clean":"2024-09-16T05:01:29.709581+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:29.709581+0000","last_undegraded":"2024-09-16T05:01:29.709581+0000","last_fullsized":"2024-09-16T05:01:29.709581+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20483,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:29.710327+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:29.710327+0000","last_peered":"2024-09-16T05:01:29.710327+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:29.710327+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":25916,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:55.153214+0000","last_change":"2024-09-16T05:01:28.328858+0000","last_active":"2024-09-16T05:01:55.153214+0000","last_peered":"2024-09-16T05:01:55.153214+0000","last_clean":"2024-09-16T04:49:35.625965+0000","last_became_active":"2024-09-16T04:49:38.237533+0000","last_became_peered":"2024-09-16T04:49:38.237533+0000","last_unstale":"2024-09-16T05:01:55.153214+0000","last_undegraded":"2024-09-16T04:49:37.763430+0000","last_fullsized":"2024-09-16T04:49:37.646127+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2935,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-466}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20268,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709303+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:29.709303+0000","last_peered":"2024-09-16T05:01:29.709303+0000","last_clean":"2024-09-16T05:01:29.709303+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:29.709303+0000","last_undegraded":"2024-09-16T05:01:29.709303+0000","last_fullsized":"2024-09-16T05:01:29.709303+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702404+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:29.702404+0000","last_peered":"2024-09-16T05:01:29.702404+0000","last_clean":"2024-09-16T05:01:29.702404+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:29.702404+0000","last_undegraded":"2024-09-16T05:01:29.702404+0000","last_fullsized":"2024-09-16T05:01:29.702404+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20332,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.851118+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:29.851118+0000","last_peered":"2024-09-16T05:01:29.851118+0000","last_clean":"2024-09-16T05:01:29.851118+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:29.851118+0000","last_undegraded":"2024-09-16T05:01:29.851118+0000","last_fullsized":"2024-09-16T05:01:29.851118+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700590+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:29.700590+0000","last_peered":"2024-09-16T05:01:29.700590+0000","last_clean":"2024-09-16T05:01:29.700590+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:29.700590+0000","last_undegraded":"2024-09-16T05:01:29.700590+0000","last_fullsized":"2024-09-16T05:01:29.700590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27325,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709306+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:29.709306+0000","last_peered":"2024-09-16T05:01:29.709306+0000","last_clean":"2024-09-16T05:01:29.709306+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:29.709306+0000","last_undegraded":"2024-09-16T05:01:29.709306+0000","last_fullsized":"2024-09-16T05:01:29.709306+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.709200+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:29.709200+0000","last_peered":"2024-09-16T05:01:29.709200+0000","last_clean":"2024-09-16T05:01:29.709200+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:29.709200+0000","last_undegraded":"2024-09-16T05:01:29.709200+0000","last_fullsized":"2024-09-16T05:01:29.709200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26861,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700743+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:29.700743+0000","last_peered":"2024-09-16T05:01:29.700743+0000","last_clean":"2024-09-16T05:01:29.700743+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:29.700743+0000","last_undegraded":"2024-09-16T05:01:29.700743+0000","last_fullsized":"2024-09-16T05:01:29.700743+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702716+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:29.702716+0000","last_peered":"2024-09-16T05:01:29.702716+0000","last_clean":"2024-09-16T05:01:29.702716+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:29.702716+0000","last_undegraded":"2024-09-16T05:01:29.702716+0000","last_fullsized":"2024-09-16T05:01:29.702716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.700310+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:29.700310+0000","last_peered":"2024-09-16T05:01:29.700310+0000","last_clean":"2024-09-16T05:01:29.700310+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:29.700310+0000","last_undegraded":"2024-09-16T05:01:29.700310+0000","last_fullsized":"2024-09-16T05:01:29.700310+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20004,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.703036+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:29.703036+0000","last_peered":"2024-09-16T05:01:29.703036+0000","last_clean":"2024-09-16T05:01:29.703036+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:29.703036+0000","last_undegraded":"2024-09-16T05:01:29.703036+0000","last_fullsized":"2024-09-16T05:01:29.703036+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702500+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:29.702500+0000","last_peered":"2024-09-16T05:01:29.702500+0000","last_clean":"2024-09-16T05:01:29.702500+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:29.702500+0000","last_undegraded":"2024-09-16T05:01:29.702500+0000","last_fullsized":"2024-09-16T05:01:29.702500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20675,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702840+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:29.702840+0000","last_peered":"2024-09-16T05:01:29.702840+0000","last_clean":"2024-09-16T05:01:29.702840+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:29.702840+0000","last_undegraded":"2024-09-16T05:01:29.702840+0000","last_fullsized":"2024-09-16T05:01:29.702840+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:01:38.101964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702677+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:29.702677+0000","last_peered":"2024-09-16T05:01:29.702677+0000","last_clean":"2024-09-16T05:01:29.702677+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:29.702677+0000","last_undegraded":"2024-09-16T05:01:29.702677+0000","last_fullsized":"2024-09-16T05:01:29.702677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27028,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702814+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:29.702814+0000","last_peered":"2024-09-16T05:01:29.702814+0000","last_clean":"2024-09-16T05:01:29.702814+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:29.702814+0000","last_undegraded":"2024-09-16T05:01:29.702814+0000","last_fullsized":"2024-09-16T05:01:29.702814+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702717+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:29.702717+0000","last_peered":"2024-09-16T05:01:29.702717+0000","last_clean":"2024-09-16T05:01:29.702717+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:29.702717+0000","last_undegraded":"2024-09-16T05:01:29.702717+0000","last_fullsized":"2024-09-16T05:01:29.702717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702704+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:29.702704+0000","last_peered":"2024-09-16T05:01:29.702704+0000","last_clean":"2024-09-16T05:01:29.702704+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:29.702704+0000","last_undegraded":"2024-09-16T05:01:29.702704+0000","last_fullsized":"2024-09-16T05:01:29.702704+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-16T05:01:29.702399+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:29.702399+0000","last_peered":"2024-09-16T05:01:29.702399+0000","last_clean":"2024-09-16T05:01:29.702399+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:29.702399+0000","last_undegraded":"2024-09-16T05:01:29.702399+0000","last_fullsized":"2024-09-16T05:01:29.702399+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33498,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166764544,"data_stored":166743670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493256,"kb_used_data":82548,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878584,"statfs":{"total":96636764160,"available":96131670016,"internally_reserved":0,"allocated":84529152,"data_stored":84176601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593876,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487588,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884252,"statfs":{"total":96636764160,"available":96137474048,"internally_reserved":0,"allocated":544768,"data_stored":181780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487812,"kb_used_data":81324,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884028,"statfs":{"total":96636764160,"available":96137244672,"internally_reserved":0,"allocated":83275776,"data_stored":82923225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:00.567 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:00.902 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:00.903 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:00.921 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-16T05:02:00.601370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34041,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476996,"kb_used_data":164424,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281638524,"statfs":{"total":289910292480,"available":288397848576,"internally_reserved":0,"allocated":168370176,"data_stored":167289590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1343998977},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1074,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002216"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20190,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:00.052930+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:00.052930+0000","last_peered":"2024-09-16T05:02:00.052930+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:00.052930+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3244}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27329,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.727913+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.727913+0000","last_peered":"2024-09-16T05:01:59.727913+0000","last_clean":"2024-09-16T05:01:59.727913+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.727913+0000","last_undegraded":"2024-09-16T05:01:59.727913+0000","last_fullsized":"2024-09-16T05:01:59.727913+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722745+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722745+0000","last_peered":"2024-09-16T05:01:59.722745+0000","last_clean":"2024-09-16T05:01:59.722745+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722745+0000","last_undegraded":"2024-09-16T05:01:59.722745+0000","last_fullsized":"2024-09-16T05:01:59.722745+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724705+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724705+0000","last_peered":"2024-09-16T05:01:59.724705+0000","last_clean":"2024-09-16T05:01:59.724705+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724705+0000","last_undegraded":"2024-09-16T05:01:59.724705+0000","last_fullsized":"2024-09-16T05:01:59.724705+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20593,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724503+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724503+0000","last_peered":"2024-09-16T05:01:59.724503+0000","last_clean":"2024-09-16T05:01:59.724503+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724503+0000","last_undegraded":"2024-09-16T05:01:59.724503+0000","last_fullsized":"2024-09-16T05:01:59.724503+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34040,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493264,"kb_used_data":82556,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878576,"statfs":{"total":96636764160,"available":96131661824,"internally_reserved":0,"allocated":84537344,"data_stored":84179468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495916,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875924,"statfs":{"total":96636764160,"available":96128946176,"internally_reserved":0,"allocated":548864,"data_stored":184030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507244206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487816,"kb_used_data":81332,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884024,"statfs":{"total":96636764160,"available":96137240576,"internally_reserved":0,"allocated":83283968,"data_stored":82926092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:00.923 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:01.236 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:01.237 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:01.255 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-16T05:02:00.601370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34041,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476996,"kb_used_data":164424,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281638524,"statfs":{"total":289910292480,"available":288397848576,"internally_reserved":0,"allocated":168370176,"data_stored":167289590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1343998977},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1074,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002216"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20190,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:00.052930+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:00.052930+0000","last_peered":"2024-09-16T05:02:00.052930+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:00.052930+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3244}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27329,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.727913+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.727913+0000","last_peered":"2024-09-16T05:01:59.727913+0000","last_clean":"2024-09-16T05:01:59.727913+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.727913+0000","last_undegraded":"2024-09-16T05:01:59.727913+0000","last_fullsized":"2024-09-16T05:01:59.727913+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722745+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722745+0000","last_peered":"2024-09-16T05:01:59.722745+0000","last_clean":"2024-09-16T05:01:59.722745+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722745+0000","last_undegraded":"2024-09-16T05:01:59.722745+0000","last_fullsized":"2024-09-16T05:01:59.722745+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724705+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724705+0000","last_peered":"2024-09-16T05:01:59.724705+0000","last_clean":"2024-09-16T05:01:59.724705+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724705+0000","last_undegraded":"2024-09-16T05:01:59.724705+0000","last_fullsized":"2024-09-16T05:01:59.724705+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20593,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724503+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724503+0000","last_peered":"2024-09-16T05:01:59.724503+0000","last_clean":"2024-09-16T05:01:59.724503+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724503+0000","last_undegraded":"2024-09-16T05:01:59.724503+0000","last_fullsized":"2024-09-16T05:01:59.724503+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34040,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493264,"kb_used_data":82556,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878576,"statfs":{"total":96636764160,"available":96131661824,"internally_reserved":0,"allocated":84537344,"data_stored":84179468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495916,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875924,"statfs":{"total":96636764160,"available":96128946176,"internally_reserved":0,"allocated":548864,"data_stored":184030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507244206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487816,"kb_used_data":81332,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884024,"statfs":{"total":96636764160,"available":96137240576,"internally_reserved":0,"allocated":83283968,"data_stored":82926092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:01.256 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:02:01.257 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20190, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:00.052930+0000', 'last_change': '2024-09-16T05:01:58.546780+0000', 'last_active': '2024-09-16T05:02:00.052930+0000', 'last_peered': '2024-09-16T05:02:00.052930+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:02:00.052930+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3244, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 76, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3244}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:02:01.257 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:01.257 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20487, 'reported_epoch': 65, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:59.725604+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:59.725604+0000', 'last_peered': '2024-09-16T05:01:59.725604+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:59.725604+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:01.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:01.627 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:01.628 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6702/18 objects degraded (37233.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":869,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1512443904,"bytes_avail":288397848576,"bytes_total":289910292480,"degraded_objects":6702,"degraded_total":18,"degraded_ratio":372.33333333333326,"recovering_objects_per_sec":134,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1074,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:01.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:02:01.648 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:01.953 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:01.953 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:01.972 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-16T05:02:00.601370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34041,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476996,"kb_used_data":164424,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281638524,"statfs":{"total":289910292480,"available":288397848576,"internally_reserved":0,"allocated":168370176,"data_stored":167289590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1343998977},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1074,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1074,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002216"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20190,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:00.052930+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:00.052930+0000","last_peered":"2024-09-16T05:02:00.052930+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:00.052930+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3244}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27329,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.727913+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.727913+0000","last_peered":"2024-09-16T05:01:59.727913+0000","last_clean":"2024-09-16T05:01:59.727913+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.727913+0000","last_undegraded":"2024-09-16T05:01:59.727913+0000","last_fullsized":"2024-09-16T05:01:59.727913+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722745+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722745+0000","last_peered":"2024-09-16T05:01:59.722745+0000","last_clean":"2024-09-16T05:01:59.722745+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722745+0000","last_undegraded":"2024-09-16T05:01:59.722745+0000","last_fullsized":"2024-09-16T05:01:59.722745+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724705+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724705+0000","last_peered":"2024-09-16T05:01:59.724705+0000","last_clean":"2024-09-16T05:01:59.724705+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724705+0000","last_undegraded":"2024-09-16T05:01:59.724705+0000","last_fullsized":"2024-09-16T05:01:59.724705+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20593,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724503+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724503+0000","last_peered":"2024-09-16T05:01:59.724503+0000","last_clean":"2024-09-16T05:01:59.724503+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724503+0000","last_undegraded":"2024-09-16T05:01:59.724503+0000","last_fullsized":"2024-09-16T05:01:59.724503+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34040,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493264,"kb_used_data":82556,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878576,"statfs":{"total":96636764160,"available":96131661824,"internally_reserved":0,"allocated":84537344,"data_stored":84179468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495916,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875924,"statfs":{"total":96636764160,"available":96128946176,"internally_reserved":0,"allocated":548864,"data_stored":184030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507244206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487816,"kb_used_data":81332,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884024,"statfs":{"total":96636764160,"available":96137240576,"internally_reserved":0,"allocated":83283968,"data_stored":82926092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:04.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:05.326 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:05.327 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:05.345 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-16T05:02:04.602616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34041,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476996,"kb_used_data":164424,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281638524,"statfs":{"total":289910292480,"available":288397848576,"internally_reserved":0,"allocated":168370176,"data_stored":167289590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1343998977},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002557"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20190,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:00.052930+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:00.052930+0000","last_peered":"2024-09-16T05:02:00.052930+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:00.052930+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3244}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27329,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.727913+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.727913+0000","last_peered":"2024-09-16T05:01:59.727913+0000","last_clean":"2024-09-16T05:01:59.727913+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.727913+0000","last_undegraded":"2024-09-16T05:01:59.727913+0000","last_fullsized":"2024-09-16T05:01:59.727913+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722745+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722745+0000","last_peered":"2024-09-16T05:01:59.722745+0000","last_clean":"2024-09-16T05:01:59.722745+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722745+0000","last_undegraded":"2024-09-16T05:01:59.722745+0000","last_fullsized":"2024-09-16T05:01:59.722745+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724705+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724705+0000","last_peered":"2024-09-16T05:01:59.724705+0000","last_clean":"2024-09-16T05:01:59.724705+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724705+0000","last_undegraded":"2024-09-16T05:01:59.724705+0000","last_fullsized":"2024-09-16T05:01:59.724705+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20593,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724503+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724503+0000","last_peered":"2024-09-16T05:01:59.724503+0000","last_clean":"2024-09-16T05:01:59.724503+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724503+0000","last_undegraded":"2024-09-16T05:01:59.724503+0000","last_fullsized":"2024-09-16T05:01:59.724503+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34040,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493264,"kb_used_data":82556,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878576,"statfs":{"total":96636764160,"available":96131661824,"internally_reserved":0,"allocated":84537344,"data_stored":84179468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495916,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875924,"statfs":{"total":96636764160,"available":96128946176,"internally_reserved":0,"allocated":548864,"data_stored":184030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507244206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487816,"kb_used_data":81332,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884024,"statfs":{"total":96636764160,"available":96137240576,"internally_reserved":0,"allocated":83283968,"data_stored":82926092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:05.347 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:05.669 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:05.669 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:05.689 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-16T05:02:04.602616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34041,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476996,"kb_used_data":164424,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281638524,"statfs":{"total":289910292480,"available":288397848576,"internally_reserved":0,"allocated":168370176,"data_stored":167289590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1343998977},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002557"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20190,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:00.052930+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:00.052930+0000","last_peered":"2024-09-16T05:02:00.052930+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:00.052930+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3244}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27329,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.727913+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.727913+0000","last_peered":"2024-09-16T05:01:59.727913+0000","last_clean":"2024-09-16T05:01:59.727913+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.727913+0000","last_undegraded":"2024-09-16T05:01:59.727913+0000","last_fullsized":"2024-09-16T05:01:59.727913+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722745+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722745+0000","last_peered":"2024-09-16T05:01:59.722745+0000","last_clean":"2024-09-16T05:01:59.722745+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722745+0000","last_undegraded":"2024-09-16T05:01:59.722745+0000","last_fullsized":"2024-09-16T05:01:59.722745+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724705+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724705+0000","last_peered":"2024-09-16T05:01:59.724705+0000","last_clean":"2024-09-16T05:01:59.724705+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724705+0000","last_undegraded":"2024-09-16T05:01:59.724705+0000","last_fullsized":"2024-09-16T05:01:59.724705+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20593,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724503+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724503+0000","last_peered":"2024-09-16T05:01:59.724503+0000","last_clean":"2024-09-16T05:01:59.724503+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724503+0000","last_undegraded":"2024-09-16T05:01:59.724503+0000","last_fullsized":"2024-09-16T05:01:59.724503+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34040,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493264,"kb_used_data":82556,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878576,"statfs":{"total":96636764160,"available":96131661824,"internally_reserved":0,"allocated":84537344,"data_stored":84179468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495916,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875924,"statfs":{"total":96636764160,"available":96128946176,"internally_reserved":0,"allocated":548864,"data_stored":184030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507244206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487816,"kb_used_data":81332,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884024,"statfs":{"total":96636764160,"available":96137240576,"internally_reserved":0,"allocated":83283968,"data_stored":82926092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:05.690 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:02:05.690 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20190, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:00.052930+0000', 'last_change': '2024-09-16T05:01:58.546780+0000', 'last_active': '2024-09-16T05:02:00.052930+0000', 'last_peered': '2024-09-16T05:02:00.052930+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:02:00.052930+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3244, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 76, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3244}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:02:05.690 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:05.690 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20487, 'reported_epoch': 65, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:59.725604+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:59.725604+0000', 'last_peered': '2024-09-16T05:01:59.725604+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:59.725604+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:05.691 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:06.095 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:06.096 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6702/18 objects degraded (37233.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":873,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1512443904,"bytes_avail":288397848576,"bytes_total":289910292480,"degraded_objects":6702,"degraded_total":18,"degraded_ratio":372.33333333333326,"recovering_objects_per_sec":67,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":542,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:06.118 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:02:06.119 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:06.433 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:06.433 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:06.453 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-16T05:02:04.602616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34041,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476996,"kb_used_data":164424,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281638524,"statfs":{"total":289910292480,"available":288397848576,"internally_reserved":0,"allocated":168370176,"data_stored":167289590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1343998977},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002557"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20190,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:00.052930+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:00.052930+0000","last_peered":"2024-09-16T05:02:00.052930+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:00.052930+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3244}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27329,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.727913+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.727913+0000","last_peered":"2024-09-16T05:01:59.727913+0000","last_clean":"2024-09-16T05:01:59.727913+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.727913+0000","last_undegraded":"2024-09-16T05:01:59.727913+0000","last_fullsized":"2024-09-16T05:01:59.727913+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722745+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722745+0000","last_peered":"2024-09-16T05:01:59.722745+0000","last_clean":"2024-09-16T05:01:59.722745+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722745+0000","last_undegraded":"2024-09-16T05:01:59.722745+0000","last_fullsized":"2024-09-16T05:01:59.722745+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724705+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724705+0000","last_peered":"2024-09-16T05:01:59.724705+0000","last_clean":"2024-09-16T05:01:59.724705+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724705+0000","last_undegraded":"2024-09-16T05:01:59.724705+0000","last_fullsized":"2024-09-16T05:01:59.724705+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20593,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724503+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724503+0000","last_peered":"2024-09-16T05:01:59.724503+0000","last_clean":"2024-09-16T05:01:59.724503+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724503+0000","last_undegraded":"2024-09-16T05:01:59.724503+0000","last_fullsized":"2024-09-16T05:01:59.724503+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34040,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493264,"kb_used_data":82556,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878576,"statfs":{"total":96636764160,"available":96131661824,"internally_reserved":0,"allocated":84537344,"data_stored":84179468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593877,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495916,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93875924,"statfs":{"total":96636764160,"available":96128946176,"internally_reserved":0,"allocated":548864,"data_stored":184030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":507244206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487816,"kb_used_data":81332,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884024,"statfs":{"total":96636764160,"available":96137240576,"internally_reserved":0,"allocated":83283968,"data_stored":82926092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:09.455 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:09.784 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:09.784 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:09.804 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-16T05:02:08.604079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34346,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460380,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1295859,"kb_avail":281655140,"statfs":{"total":289910292480,"available":288414863360,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1326959617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-847,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.899158"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20800,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:05.062295+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:05.062295+0000","last_peered":"2024-09-16T05:02:05.062295+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:05.062295+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2939,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2939}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722745+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722745+0000","last_peered":"2024-09-16T05:01:59.722745+0000","last_clean":"2024-09-16T05:01:59.722745+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722745+0000","last_undegraded":"2024-09-16T05:01:59.722745+0000","last_fullsized":"2024-09-16T05:01:59.722745+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724705+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724705+0000","last_peered":"2024-09-16T05:01:59.724705+0000","last_clean":"2024-09-16T05:01:59.724705+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724705+0000","last_undegraded":"2024-09-16T05:01:59.724705+0000","last_fullsized":"2024-09-16T05:01:59.724705+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34345,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479280,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892560,"statfs":{"total":96636764160,"available":96145981440,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:09.806 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:10.119 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:10.119 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:10.138 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-16T05:02:08.604079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34346,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1460380,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1295859,"kb_avail":281655140,"statfs":{"total":289910292480,"available":288414863360,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1326959617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-847,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.899158"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20800,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:05.062295+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:05.062295+0000","last_peered":"2024-09-16T05:02:05.062295+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:05.062295+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2939,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2939}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722745+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722745+0000","last_peered":"2024-09-16T05:01:59.722745+0000","last_clean":"2024-09-16T05:01:59.722745+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722745+0000","last_undegraded":"2024-09-16T05:01:59.722745+0000","last_fullsized":"2024-09-16T05:01:59.722745+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724705+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724705+0000","last_peered":"2024-09-16T05:01:59.724705+0000","last_clean":"2024-09-16T05:01:59.724705+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724705+0000","last_undegraded":"2024-09-16T05:01:59.724705+0000","last_fullsized":"2024-09-16T05:01:59.724705+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34345,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479280,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892560,"statfs":{"total":96636764160,"available":96145981440,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":490204846},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:10.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:02:10.139 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 20800, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:05.062295+0000', 'last_change': '2024-09-16T05:01:58.546780+0000', 'last_active': '2024-09-16T05:02:05.062295+0000', 'last_peered': '2024-09-16T05:02:05.062295+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:02:05.062295+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2939, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 381, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:02:10.140 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:10.140 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20487, 'reported_epoch': 65, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:59.725604+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:59.725604+0000', 'last_peered': '2024-09-16T05:01:59.725604+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:59.725604+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:10.140 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:10.560 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:10.560 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6397/18 objects degraded (35538.889%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":878,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1495429120,"bytes_avail":288414863360,"bytes_total":289910292480,"degraded_objects":6397,"degraded_total":18,"degraded_ratio":355.38888888888891,"recovering_objects_per_sec":85,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":847,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:10.580 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:02:10.580 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:10.906 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:10.906 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:10.926 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2024-09-16T05:02:10.604594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34679,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464540,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281650980,"statfs":{"total":289910292480,"available":288410603520,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1180,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.884297"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":21466,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:10.055004+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:10.055004+0000","last_peered":"2024-09-16T05:02:10.055004+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:10.055004+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":714,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2606}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34678,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593879,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483440,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888400,"statfs":{"total":96636764160,"available":96141721600,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:13.928 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:14.242 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:14.243 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:14.260 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-16T05:02:12.605069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34679,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464540,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281650980,"statfs":{"total":289910292480,"available":288410603520,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003699"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":21466,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:10.055004+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:10.055004+0000","last_peered":"2024-09-16T05:02:10.055004+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:10.055004+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":714,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2606}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34678,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593879,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483440,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888400,"statfs":{"total":96636764160,"available":96141721600,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:14.262 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:14.564 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:14.583 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-16T05:02:12.605069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34679,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464540,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281650980,"statfs":{"total":289910292480,"available":288410603520,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003699"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":21466,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:10.055004+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:10.055004+0000","last_peered":"2024-09-16T05:02:10.055004+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:10.055004+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":714,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2606}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34678,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593879,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483440,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888400,"statfs":{"total":96636764160,"available":96141721600,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:14.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:02:14.584 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 21466, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:10.055004+0000', 'last_change': '2024-09-16T05:01:58.546780+0000', 'last_active': '2024-09-16T05:02:10.055004+0000', 'last_peered': '2024-09-16T05:02:10.055004+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:02:10.055004+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2606, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 714, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2606}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:02:14.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:14.585 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20487, 'reported_epoch': 65, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:59.725604+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:59.725604+0000', 'last_peered': '2024-09-16T05:01:59.725604+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:59.725604+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:14.585 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:14.954 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:14.954 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6064/18 objects degraded (33688.889%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":882,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1499688960,"bytes_avail":288410603520,"bytes_total":289910292480,"degraded_objects":6064,"degraded_total":18,"degraded_ratio":336.88888888888891,"recovering_objects_per_sec":53,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":638,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:14.974 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:02:14.974 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:15.305 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:15.306 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:15.323 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2024-09-16T05:02:14.605606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34679,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464540,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281650980,"statfs":{"total":289910292480,"available":288410603520,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1331219457},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003549"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":21466,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:10.055004+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:10.055004+0000","last_peered":"2024-09-16T05:02:10.055004+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:10.055004+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2606,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":714,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2606}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34678,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593879,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483440,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888400,"statfs":{"total":96636764160,"available":96141721600,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":494464686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:18.326 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:18.643 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:18.662 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2024-09-16T05:02:18.606734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5712,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35031,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468700,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646820,"statfs":{"total":289910292480,"available":288406343680,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-685,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":685,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003474"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":22170,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:15.067799+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:15.067799+0000","last_peered":"2024-09-16T05:02:15.067799+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:15.067799+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1066,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2254}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5712,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35030,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593880,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487600,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884240,"statfs":{"total":96636764160,"available":96137461760,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:18.664 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:18.967 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:18.985 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2024-09-16T05:02:18.606734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5712,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35031,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468700,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646820,"statfs":{"total":289910292480,"available":288406343680,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-685,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":685,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003474"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":22170,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:15.067799+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:15.067799+0000","last_peered":"2024-09-16T05:02:15.067799+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:15.067799+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1066,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2254}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5712,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35030,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593880,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487600,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884240,"statfs":{"total":96636764160,"available":96137461760,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:18.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:02:18.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 22170, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:15.067799+0000', 'last_change': '2024-09-16T05:01:58.546780+0000', 'last_active': '2024-09-16T05:02:15.067799+0000', 'last_peered': '2024-09-16T05:02:15.067799+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:02:15.067799+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2254, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1066, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2254}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:02:18.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:18.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20487, 'reported_epoch': 65, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:59.725604+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:59.725604+0000', 'last_peered': '2024-09-16T05:01:59.725604+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:59.725604+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:18.987 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:19.357 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:19.357 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5712/18 objects degraded (31733.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":887,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1503948800,"bytes_avail":288406343680,"bytes_total":289910292480,"degraded_objects":5712,"degraded_total":18,"degraded_ratio":317.33333333333331,"recovering_objects_per_sec":57,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":685,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:19.377 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:02:19.377 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:19.731 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:19.731 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:19.753 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2024-09-16T05:02:18.606734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5712,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35031,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468700,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646820,"statfs":{"total":289910292480,"available":288406343680,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-685,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":685,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003474"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20150,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721713+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721713+0000","last_peered":"2024-09-16T05:01:59.721713+0000","last_clean":"2024-09-16T05:01:59.721713+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721713+0000","last_undegraded":"2024-09-16T05:01:59.721713+0000","last_fullsized":"2024-09-16T05:01:59.721713+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":22170,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:15.067799+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:15.067799+0000","last_peered":"2024-09-16T05:02:15.067799+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:15.067799+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1066,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2254}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20454,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721473+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721473+0000","last_peered":"2024-09-16T05:01:59.721473+0000","last_clean":"2024-09-16T05:01:59.721473+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721473+0000","last_undegraded":"2024-09-16T05:01:59.721473+0000","last_fullsized":"2024-09-16T05:01:59.721473+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5712,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35030,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593880,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487600,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884240,"statfs":{"total":96636764160,"available":96137461760,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:22.755 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:23.086 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:23.086 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:23.106 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-09-16T05:02:22.607921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35426,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468700,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646820,"statfs":{"total":289910292480,"available":288406343680,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":747,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003326"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721842+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721842+0000","last_peered":"2024-09-16T05:01:59.721842+0000","last_clean":"2024-09-16T05:01:59.721842+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721842+0000","last_undegraded":"2024-09-16T05:01:59.721842+0000","last_fullsized":"2024-09-16T05:01:59.721842+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":22961,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:20.074867+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:20.074867+0000","last_peered":"2024-09-16T05:02:20.074867+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:20.074867+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1859}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20455,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721540+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721540+0000","last_peered":"2024-09-16T05:01:59.721540+0000","last_clean":"2024-09-16T05:01:59.721540+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721540+0000","last_undegraded":"2024-09-16T05:01:59.721540+0000","last_fullsized":"2024-09-16T05:01:59.721540+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35425,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487600,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884240,"statfs":{"total":96636764160,"available":96137461760,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:23.107 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:23.430 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:23.449 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-09-16T05:02:22.607921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35426,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468700,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646820,"statfs":{"total":289910292480,"available":288406343680,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":747,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003326"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721842+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721842+0000","last_peered":"2024-09-16T05:01:59.721842+0000","last_clean":"2024-09-16T05:01:59.721842+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721842+0000","last_undegraded":"2024-09-16T05:01:59.721842+0000","last_fullsized":"2024-09-16T05:01:59.721842+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":22961,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:20.074867+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:20.074867+0000","last_peered":"2024-09-16T05:02:20.074867+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:20.074867+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1859}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20455,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721540+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721540+0000","last_peered":"2024-09-16T05:01:59.721540+0000","last_clean":"2024-09-16T05:01:59.721540+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721540+0000","last_undegraded":"2024-09-16T05:01:59.721540+0000","last_fullsized":"2024-09-16T05:01:59.721540+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35425,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487600,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884240,"statfs":{"total":96636764160,"available":96137461760,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:02:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 22961, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:20.074867+0000', 'last_change': '2024-09-16T05:01:58.546780+0000', 'last_active': '2024-09-16T05:02:20.074867+0000', 'last_peered': '2024-09-16T05:02:20.074867+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:02:20.074867+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1859, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1461, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1859}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:02:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:23.450 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20487, 'reported_epoch': 65, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:59.725604+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:59.725604+0000', 'last_peered': '2024-09-16T05:01:59.725604+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:59.725604+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:23.451 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:23.889 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5317/18 objects degraded (29538.889%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":891,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1503948800,"bytes_avail":288406343680,"bytes_total":289910292480,"degraded_objects":5317,"degraded_total":18,"degraded_ratio":295.38888888888891,"recovering_objects_per_sec":62,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":747,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:23.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:02:23.909 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:24.229 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:24.230 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:24.249 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-09-16T05:02:22.607921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35426,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468700,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281646820,"statfs":{"total":289910292480,"available":288406343680,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1335479297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":747,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003326"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721842+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721842+0000","last_peered":"2024-09-16T05:01:59.721842+0000","last_clean":"2024-09-16T05:01:59.721842+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721842+0000","last_undegraded":"2024-09-16T05:01:59.721842+0000","last_fullsized":"2024-09-16T05:01:59.721842+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":22961,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:20.074867+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:20.074867+0000","last_peered":"2024-09-16T05:02:20.074867+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:20.074867+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1859}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20455,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721540+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721540+0000","last_peered":"2024-09-16T05:01:59.721540+0000","last_clean":"2024-09-16T05:01:59.721540+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721540+0000","last_undegraded":"2024-09-16T05:01:59.721540+0000","last_fullsized":"2024-09-16T05:01:59.721540+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722486+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722486+0000","last_peered":"2024-09-16T05:01:59.722486+0000","last_clean":"2024-09-16T05:01:59.722486+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722486+0000","last_undegraded":"2024-09-16T05:01:59.722486+0000","last_fullsized":"2024-09-16T05:01:59.722486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35425,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593881,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487600,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884240,"statfs":{"total":96636764160,"available":96137461760,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":498724526},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:27.251 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:27.568 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:27.568 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:27.588 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-16T05:02:26.609379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4906,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35837,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472860,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642660,"statfs":{"total":289910292480,"available":288402083840,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1158,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003773"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721842+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721842+0000","last_peered":"2024-09-16T05:01:59.721842+0000","last_clean":"2024-09-16T05:01:59.721842+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721842+0000","last_undegraded":"2024-09-16T05:01:59.721842+0000","last_fullsized":"2024-09-16T05:01:59.721842+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":23783,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:25.070399+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:25.070399+0000","last_peered":"2024-09-16T05:02:25.070399+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:25.070399+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1872,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1448}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20455,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721540+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721540+0000","last_peered":"2024-09-16T05:01:59.721540+0000","last_clean":"2024-09-16T05:01:59.721540+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721540+0000","last_undegraded":"2024-09-16T05:01:59.721540+0000","last_fullsized":"2024-09-16T05:01:59.721540+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722611+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722611+0000","last_peered":"2024-09-16T05:01:59.722611+0000","last_clean":"2024-09-16T05:01:59.722611+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722611+0000","last_undegraded":"2024-09-16T05:01:59.722611+0000","last_fullsized":"2024-09-16T05:01:59.722611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4906,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35836,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593882,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491760,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880080,"statfs":{"total":96636764160,"available":96133201920,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:27.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:27.936 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:27.957 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-16T05:02:26.609379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4906,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35837,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472860,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642660,"statfs":{"total":289910292480,"available":288402083840,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1158,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003773"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721842+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721842+0000","last_peered":"2024-09-16T05:01:59.721842+0000","last_clean":"2024-09-16T05:01:59.721842+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721842+0000","last_undegraded":"2024-09-16T05:01:59.721842+0000","last_fullsized":"2024-09-16T05:01:59.721842+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":23783,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:25.070399+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:25.070399+0000","last_peered":"2024-09-16T05:02:25.070399+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:25.070399+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1872,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1448}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20455,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721540+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721540+0000","last_peered":"2024-09-16T05:01:59.721540+0000","last_clean":"2024-09-16T05:01:59.721540+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721540+0000","last_undegraded":"2024-09-16T05:01:59.721540+0000","last_fullsized":"2024-09-16T05:01:59.721540+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722611+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722611+0000","last_peered":"2024-09-16T05:01:59.722611+0000","last_clean":"2024-09-16T05:01:59.722611+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722611+0000","last_undegraded":"2024-09-16T05:01:59.722611+0000","last_fullsized":"2024-09-16T05:01:59.722611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4906,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35836,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593882,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491760,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880080,"statfs":{"total":96636764160,"available":96133201920,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:27.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:02:27.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 23783, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:25.070399+0000', 'last_change': '2024-09-16T05:01:58.546780+0000', 'last_active': '2024-09-16T05:02:25.070399+0000', 'last_peered': '2024-09-16T05:02:25.070399+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:02:25.070399+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1448, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1872, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1448}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:02:27.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:27.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20487, 'reported_epoch': 65, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:59.725604+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:59.725604+0000', 'last_peered': '2024-09-16T05:01:59.725604+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:59.725604+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:27.959 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:28.348 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:28.348 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4906/18 objects degraded (27255.556%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":896,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1508208640,"bytes_avail":288402083840,"bytes_total":289910292480,"degraded_objects":4906,"degraded_total":18,"degraded_ratio":272.55555555555554,"recovering_objects_per_sec":96,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1158,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:28.367 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:02:28.367 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:28.702 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:28.703 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:28.721 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-16T05:02:28.610207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4906,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35837,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472860,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281642660,"statfs":{"total":289910292480,"available":288402083840,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1339739137},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-806,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":806,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004201"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721842+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721842+0000","last_peered":"2024-09-16T05:01:59.721842+0000","last_clean":"2024-09-16T05:01:59.721842+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721842+0000","last_undegraded":"2024-09-16T05:01:59.721842+0000","last_fullsized":"2024-09-16T05:01:59.721842+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":23783,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:25.070399+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:25.070399+0000","last_peered":"2024-09-16T05:02:25.070399+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:25.070399+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1872,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1448}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20455,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721540+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721540+0000","last_peered":"2024-09-16T05:01:59.721540+0000","last_clean":"2024-09-16T05:01:59.721540+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721540+0000","last_undegraded":"2024-09-16T05:01:59.721540+0000","last_fullsized":"2024-09-16T05:01:59.721540+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20272,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725506+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725506+0000","last_peered":"2024-09-16T05:01:59.725506+0000","last_clean":"2024-09-16T05:01:59.725506+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725506+0000","last_undegraded":"2024-09-16T05:01:59.725506+0000","last_fullsized":"2024-09-16T05:01:59.725506+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27032,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722666+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722666+0000","last_peered":"2024-09-16T05:01:59.722666+0000","last_clean":"2024-09-16T05:01:59.722666+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722666+0000","last_undegraded":"2024-09-16T05:01:59.722666+0000","last_fullsized":"2024-09-16T05:01:59.722666+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722611+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722611+0000","last_peered":"2024-09-16T05:01:59.722611+0000","last_clean":"2024-09-16T05:01:59.722611+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722611+0000","last_undegraded":"2024-09-16T05:01:59.722611+0000","last_fullsized":"2024-09-16T05:01:59.722611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4906,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35836,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593882,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491760,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880080,"statfs":{"total":96636764160,"available":96133201920,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":502984366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:31.724 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:32.071 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:32.089 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-09-16T05:02:30.610662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36287,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478428,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1313907,"kb_avail":281637092,"statfs":{"total":289910292480,"available":288396382208,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1345440769},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1256,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003928"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721842+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721842+0000","last_peered":"2024-09-16T05:01:59.721842+0000","last_clean":"2024-09-16T05:01:59.721842+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721842+0000","last_undegraded":"2024-09-16T05:01:59.721842+0000","last_fullsized":"2024-09-16T05:01:59.721842+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":24683,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:30.077539+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:30.077539+0000","last_peered":"2024-09-16T05:02:30.077539+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:30.077539+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-998}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20455,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721540+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721540+0000","last_peered":"2024-09-16T05:01:59.721540+0000","last_clean":"2024-09-16T05:01:59.721540+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721540+0000","last_undegraded":"2024-09-16T05:01:59.721540+0000","last_fullsized":"2024-09-16T05:01:59.721540+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20273,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725649+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725649+0000","last_peered":"2024-09-16T05:01:59.725649+0000","last_clean":"2024-09-16T05:01:59.725649+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725649+0000","last_undegraded":"2024-09-16T05:01:59.725649+0000","last_fullsized":"2024-09-16T05:01:59.725649+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27033,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722746+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722746+0000","last_peered":"2024-09-16T05:01:59.722746+0000","last_clean":"2024-09-16T05:01:59.722746+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722746+0000","last_undegraded":"2024-09-16T05:01:59.722746+0000","last_fullsized":"2024-09-16T05:01:59.722746+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722611+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722611+0000","last_peered":"2024-09-16T05:01:59.722611+0000","last_clean":"2024-09-16T05:01:59.722611+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722611+0000","last_undegraded":"2024-09-16T05:01:59.722611+0000","last_fullsized":"2024-09-16T05:01:59.722611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36286,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593883,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497328,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874512,"statfs":{"total":96636764160,"available":96127500288,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:32.090 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:32.426 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:32.444 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-09-16T05:02:30.610662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36287,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478428,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1313907,"kb_avail":281637092,"statfs":{"total":289910292480,"available":288396382208,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1345440769},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1256,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1256,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003928"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721842+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721842+0000","last_peered":"2024-09-16T05:01:59.721842+0000","last_clean":"2024-09-16T05:01:59.721842+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721842+0000","last_undegraded":"2024-09-16T05:01:59.721842+0000","last_fullsized":"2024-09-16T05:01:59.721842+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":24683,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:30.077539+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:30.077539+0000","last_peered":"2024-09-16T05:02:30.077539+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:30.077539+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-998}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20455,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721540+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721540+0000","last_peered":"2024-09-16T05:01:59.721540+0000","last_clean":"2024-09-16T05:01:59.721540+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721540+0000","last_undegraded":"2024-09-16T05:01:59.721540+0000","last_fullsized":"2024-09-16T05:01:59.721540+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20273,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725649+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725649+0000","last_peered":"2024-09-16T05:01:59.725649+0000","last_clean":"2024-09-16T05:01:59.725649+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725649+0000","last_undegraded":"2024-09-16T05:01:59.725649+0000","last_fullsized":"2024-09-16T05:01:59.725649+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27033,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722746+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722746+0000","last_peered":"2024-09-16T05:01:59.722746+0000","last_clean":"2024-09-16T05:01:59.722746+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722746+0000","last_undegraded":"2024-09-16T05:01:59.722746+0000","last_fullsized":"2024-09-16T05:01:59.722746+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722611+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722611+0000","last_peered":"2024-09-16T05:01:59.722611+0000","last_clean":"2024-09-16T05:01:59.722611+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722611+0000","last_undegraded":"2024-09-16T05:01:59.722611+0000","last_fullsized":"2024-09-16T05:01:59.722611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36286,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593883,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497328,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874512,"statfs":{"total":96636764160,"available":96127500288,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:32.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:02:32.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 24683, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:30.077539+0000', 'last_change': '2024-09-16T05:01:58.546780+0000', 'last_active': '2024-09-16T05:02:30.077539+0000', 'last_peered': '2024-09-16T05:02:30.077539+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:02:30.077539+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 998, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2322, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -998}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:02:32.445 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:32.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20487, 'reported_epoch': 65, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:59.725604+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:59.725604+0000', 'last_peered': '2024-09-16T05:01:59.725604+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:59.725604+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:32.446 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:32.818 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:32.818 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4456/18 objects degraded (24755.556%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":900,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1513910272,"bytes_avail":288396382208,"bytes_total":289910292480,"degraded_objects":4456,"degraded_total":18,"degraded_ratio":247.55555555555554,"recovering_objects_per_sec":71,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":861,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:32.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:02:32.839 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:33.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:33.146 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:33.168 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-09-16T05:02:32.611298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36287,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478428,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1313907,"kb_avail":281637092,"statfs":{"total":289910292480,"available":288396382208,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1345440769},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":861,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004100"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721842+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721842+0000","last_peered":"2024-09-16T05:01:59.721842+0000","last_clean":"2024-09-16T05:01:59.721842+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721842+0000","last_undegraded":"2024-09-16T05:01:59.721842+0000","last_fullsized":"2024-09-16T05:01:59.721842+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":24683,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:30.077539+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:30.077539+0000","last_peered":"2024-09-16T05:02:30.077539+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:30.077539+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-998}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20455,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721540+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721540+0000","last_peered":"2024-09-16T05:01:59.721540+0000","last_clean":"2024-09-16T05:01:59.721540+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721540+0000","last_undegraded":"2024-09-16T05:01:59.721540+0000","last_fullsized":"2024-09-16T05:01:59.721540+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20273,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725649+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725649+0000","last_peered":"2024-09-16T05:01:59.725649+0000","last_clean":"2024-09-16T05:01:59.725649+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725649+0000","last_undegraded":"2024-09-16T05:01:59.725649+0000","last_fullsized":"2024-09-16T05:01:59.725649+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27033,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722746+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722746+0000","last_peered":"2024-09-16T05:01:59.722746+0000","last_clean":"2024-09-16T05:01:59.722746+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722746+0000","last_undegraded":"2024-09-16T05:01:59.722746+0000","last_fullsized":"2024-09-16T05:01:59.722746+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722611+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722611+0000","last_peered":"2024-09-16T05:01:59.722611+0000","last_clean":"2024-09-16T05:01:59.722611+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722611+0000","last_undegraded":"2024-09-16T05:01:59.722611+0000","last_fullsized":"2024-09-16T05:01:59.722611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36286,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593883,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497328,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874512,"statfs":{"total":96636764160,"available":96127500288,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:36.171 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:36.484 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:36.502 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-09-16T05:02:34.611663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36287,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478428,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1313907,"kb_avail":281637092,"statfs":{"total":289910292480,"available":288396382208,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1345440769},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-861,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":861,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003742"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721842+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721842+0000","last_peered":"2024-09-16T05:01:59.721842+0000","last_clean":"2024-09-16T05:01:59.721842+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721842+0000","last_undegraded":"2024-09-16T05:01:59.721842+0000","last_fullsized":"2024-09-16T05:01:59.721842+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":24683,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:30.077539+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:30.077539+0000","last_peered":"2024-09-16T05:02:30.077539+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:30.077539+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-998}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20455,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721540+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721540+0000","last_peered":"2024-09-16T05:01:59.721540+0000","last_clean":"2024-09-16T05:01:59.721540+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721540+0000","last_undegraded":"2024-09-16T05:01:59.721540+0000","last_fullsized":"2024-09-16T05:01:59.721540+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20273,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725649+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725649+0000","last_peered":"2024-09-16T05:01:59.725649+0000","last_clean":"2024-09-16T05:01:59.725649+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725649+0000","last_undegraded":"2024-09-16T05:01:59.725649+0000","last_fullsized":"2024-09-16T05:01:59.725649+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27033,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722746+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722746+0000","last_peered":"2024-09-16T05:01:59.722746+0000","last_clean":"2024-09-16T05:01:59.722746+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722746+0000","last_undegraded":"2024-09-16T05:01:59.722746+0000","last_fullsized":"2024-09-16T05:01:59.722746+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722611+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722611+0000","last_peered":"2024-09-16T05:01:59.722611+0000","last_clean":"2024-09-16T05:01:59.722611+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722611+0000","last_undegraded":"2024-09-16T05:01:59.722611+0000","last_fullsized":"2024-09-16T05:01:59.722611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4456,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36286,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593883,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497328,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874512,"statfs":{"total":96636764160,"available":96127500288,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:36.504 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:36.806 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:36.823 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2024-09-16T05:02:36.612190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3919,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36824,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482588,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1318067,"kb_avail":281632932,"statfs":{"total":289910292480,"available":288392122368,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1349700609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1398,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003528"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721842+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721842+0000","last_peered":"2024-09-16T05:01:59.721842+0000","last_clean":"2024-09-16T05:01:59.721842+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721842+0000","last_undegraded":"2024-09-16T05:01:59.721842+0000","last_fullsized":"2024-09-16T05:01:59.721842+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":25757,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:35.082707+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:35.082707+0000","last_peered":"2024-09-16T05:02:35.082707+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:35.082707+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2859,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-461}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20455,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721540+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721540+0000","last_peered":"2024-09-16T05:01:59.721540+0000","last_clean":"2024-09-16T05:01:59.721540+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721540+0000","last_undegraded":"2024-09-16T05:01:59.721540+0000","last_fullsized":"2024-09-16T05:01:59.721540+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20273,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725649+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725649+0000","last_peered":"2024-09-16T05:01:59.725649+0000","last_clean":"2024-09-16T05:01:59.725649+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725649+0000","last_undegraded":"2024-09-16T05:01:59.725649+0000","last_fullsized":"2024-09-16T05:01:59.725649+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27033,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722746+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722746+0000","last_peered":"2024-09-16T05:01:59.722746+0000","last_clean":"2024-09-16T05:01:59.722746+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722746+0000","last_undegraded":"2024-09-16T05:01:59.722746+0000","last_fullsized":"2024-09-16T05:01:59.722746+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722611+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722611+0000","last_peered":"2024-09-16T05:01:59.722611+0000","last_clean":"2024-09-16T05:01:59.722611+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722611+0000","last_undegraded":"2024-09-16T05:01:59.722611+0000","last_fullsized":"2024-09-16T05:01:59.722611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3919,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36823,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593884,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501488,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870352,"statfs":{"total":96636764160,"available":96123240448,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:36.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-16T05:02:36.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "51'19858", 'reported_seq': 25757, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:35.082707+0000', 'last_change': '2024-09-16T05:01:58.546780+0000', 'last_active': '2024-09-16T05:02:35.082707+0000', 'last_peered': '2024-09-16T05:02:35.082707+0000', 'last_clean': '2024-09-16T04:49:35.615806+0000', 'last_became_active': '2024-09-16T04:49:38.228266+0000', 'last_became_peered': '2024-09-16T04:49:38.228266+0000', 'last_unstale': '2024-09-16T05:02:35.082707+0000', 'last_undegraded': '2024-09-16T04:49:37.780333+0000', 'last_fullsized': '2024-09-16T04:49:37.652331+0000', 'mapping_epoch': 44, 'log_start': "47'9800", 'ondisk_log_start': "47'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "40'3261", 'last_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:33.068296+0000', 'objects_scrubbed': 3257, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 707, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 461, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16597, 'num_write_kb': 26677, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2859, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -461}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T05:02:36.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:36.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20487, 'reported_epoch': 65, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:01:59.725604+0000', 'last_change': '2024-09-16T04:49:38.301521+0000', 'last_active': '2024-09-16T05:01:59.725604+0000', 'last_peered': '2024-09-16T05:01:59.725604+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:01:59.725604+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3458, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:36.825 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:37.205 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:37.205 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3919/18 objects degraded (21772.222%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":904,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1518170112,"bytes_avail":288392122368,"bytes_total":289910292480,"degraded_objects":3919,"degraded_total":18,"degraded_ratio":217.72222222222223,"recovering_objects_per_sec":116,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1398,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:37.224 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:02:37.224 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:37.536 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:37.557 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2024-09-16T05:02:36.612190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3919,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36824,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482588,"kb_used_data":164448,"kb_used_omap":12,"kb_used_meta":1318067,"kb_avail":281632932,"statfs":{"total":289910292480,"available":288392122368,"internally_reserved":0,"allocated":168394752,"data_stored":167298128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1349700609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1398,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003528"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26547,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728547+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:01:59.728547+0000","last_peered":"2024-09-16T05:01:59.728547+0000","last_clean":"2024-09-16T05:01:59.728547+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:01:59.728547+0000","last_undegraded":"2024-09-16T05:01:59.728547+0000","last_fullsized":"2024-09-16T05:01:59.728547+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721842+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:01:59.721842+0000","last_peered":"2024-09-16T05:01:59.721842+0000","last_clean":"2024-09-16T05:01:59.721842+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:01:59.721842+0000","last_undegraded":"2024-09-16T05:01:59.721842+0000","last_fullsized":"2024-09-16T05:01:59.721842+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":25757,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:35.082707+0000","last_change":"2024-09-16T05:01:58.546780+0000","last_active":"2024-09-16T05:02:35.082707+0000","last_peered":"2024-09-16T05:02:35.082707+0000","last_clean":"2024-09-16T04:49:35.615806+0000","last_became_active":"2024-09-16T04:49:38.228266+0000","last_became_peered":"2024-09-16T04:49:38.228266+0000","last_unstale":"2024-09-16T05:02:35.082707+0000","last_undegraded":"2024-09-16T04:49:37.780333+0000","last_fullsized":"2024-09-16T04:49:37.652331+0000","mapping_epoch":44,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2859,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-461}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20455,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721540+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:01:59.721540+0000","last_peered":"2024-09-16T05:01:59.721540+0000","last_clean":"2024-09-16T05:01:59.721540+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:01:59.721540+0000","last_undegraded":"2024-09-16T05:01:59.721540+0000","last_fullsized":"2024-09-16T05:01:59.721540+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20403,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728198+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:01:59.728198+0000","last_peered":"2024-09-16T05:01:59.728198+0000","last_clean":"2024-09-16T05:01:59.728198+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:01:59.728198+0000","last_undegraded":"2024-09-16T05:01:59.728198+0000","last_fullsized":"2024-09-16T05:01:59.728198+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20487,"reported_epoch":65,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T05:01:59.725604+0000","last_change":"2024-09-16T04:49:38.301521+0000","last_active":"2024-09-16T05:01:59.725604+0000","last_peered":"2024-09-16T05:01:59.725604+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:01:59.725604+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20028,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.945469+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:01:59.945469+0000","last_peered":"2024-09-16T05:01:59.945469+0000","last_clean":"2024-09-16T05:01:59.945469+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:01:59.945469+0000","last_undegraded":"2024-09-16T05:01:59.945469+0000","last_fullsized":"2024-09-16T05:01:59.945469+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20273,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725649+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:01:59.725649+0000","last_peered":"2024-09-16T05:01:59.725649+0000","last_clean":"2024-09-16T05:01:59.725649+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:01:59.725649+0000","last_undegraded":"2024-09-16T05:01:59.725649+0000","last_fullsized":"2024-09-16T05:01:59.725649+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722312+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:01:59.722312+0000","last_peered":"2024-09-16T05:01:59.722312+0000","last_clean":"2024-09-16T05:01:59.722312+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:01:59.722312+0000","last_undegraded":"2024-09-16T05:01:59.722312+0000","last_fullsized":"2024-09-16T05:01:59.722312+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20336,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723022+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:01:59.723022+0000","last_peered":"2024-09-16T05:01:59.723022+0000","last_clean":"2024-09-16T05:01:59.723022+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:01:59.723022+0000","last_undegraded":"2024-09-16T05:01:59.723022+0000","last_fullsized":"2024-09-16T05:01:59.723022+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.723057+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:01:59.723057+0000","last_peered":"2024-09-16T05:01:59.723057+0000","last_clean":"2024-09-16T05:01:59.723057+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:01:59.723057+0000","last_undegraded":"2024-09-16T05:01:59.723057+0000","last_fullsized":"2024-09-16T05:01:59.723057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27330,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.728538+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:01:59.728538+0000","last_peered":"2024-09-16T05:01:59.728538+0000","last_clean":"2024-09-16T05:01:59.728538+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:01:59.728538+0000","last_undegraded":"2024-09-16T05:01:59.728538+0000","last_fullsized":"2024-09-16T05:01:59.728538+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":139,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725509+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:01:59.725509+0000","last_peered":"2024-09-16T05:01:59.725509+0000","last_clean":"2024-09-16T05:01:59.725509+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:01:59.725509+0000","last_undegraded":"2024-09-16T05:01:59.725509+0000","last_fullsized":"2024-09-16T05:01:59.725509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26865,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721963+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:01:59.721963+0000","last_peered":"2024-09-16T05:01:59.721963+0000","last_clean":"2024-09-16T05:01:59.721963+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:01:59.721963+0000","last_undegraded":"2024-09-16T05:01:59.721963+0000","last_fullsized":"2024-09-16T05:01:59.721963+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:00.066185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722871+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:01:59.722871+0000","last_peered":"2024-09-16T05:01:59.722871+0000","last_clean":"2024-09-16T05:01:59.722871+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:01:59.722871+0000","last_undegraded":"2024-09-16T05:01:59.722871+0000","last_fullsized":"2024-09-16T05:01:59.722871+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26875,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.721100+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:01:59.721100+0000","last_peered":"2024-09-16T05:01:59.721100+0000","last_clean":"2024-09-16T05:01:59.721100+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:01:59.721100+0000","last_undegraded":"2024-09-16T05:01:59.721100+0000","last_fullsized":"2024-09-16T05:01:59.721100+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20008,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722602+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:01:59.722602+0000","last_peered":"2024-09-16T05:01:59.722602+0000","last_clean":"2024-09-16T05:01:59.722602+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:01:59.722602+0000","last_undegraded":"2024-09-16T05:01:59.722602+0000","last_fullsized":"2024-09-16T05:01:59.722602+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:02.250195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724804+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:01:59.724804+0000","last_peered":"2024-09-16T05:01:59.724804+0000","last_clean":"2024-09-16T05:01:59.724804+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:01:59.724804+0000","last_undegraded":"2024-09-16T05:01:59.724804+0000","last_fullsized":"2024-09-16T05:01:59.724804+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20679,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722788+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:01:59.722788+0000","last_peered":"2024-09-16T05:01:59.722788+0000","last_clean":"2024-09-16T05:01:59.722788+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:01:59.722788+0000","last_undegraded":"2024-09-16T05:01:59.722788+0000","last_fullsized":"2024-09-16T05:01:59.722788+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:03.274382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.725259+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:01:59.725259+0000","last_peered":"2024-09-16T05:01:59.725259+0000","last_clean":"2024-09-16T05:01:59.725259+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:01:59.725259+0000","last_undegraded":"2024-09-16T05:01:59.725259+0000","last_fullsized":"2024-09-16T05:01:59.725259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27033,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722746+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:01:59.722746+0000","last_peered":"2024-09-16T05:01:59.722746+0000","last_clean":"2024-09-16T05:01:59.722746+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:01:59.722746+0000","last_undegraded":"2024-09-16T05:01:59.722746+0000","last_fullsized":"2024-09-16T05:01:59.722746+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722611+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:01:59.722611+0000","last_peered":"2024-09-16T05:01:59.722611+0000","last_clean":"2024-09-16T05:01:59.722611+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:01:59.722611+0000","last_undegraded":"2024-09-16T05:01:59.722611+0000","last_fullsized":"2024-09-16T05:01:59.722611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":109,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.722462+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:01:59.722462+0000","last_peered":"2024-09-16T05:01:59.722462+0000","last_clean":"2024-09-16T05:01:59.722462+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:01:59.722462+0000","last_undegraded":"2024-09-16T05:01:59.722462+0000","last_fullsized":"2024-09-16T05:01:59.722462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20594,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-16T05:01:59.724581+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:01:59.724581+0000","last_peered":"2024-09-16T05:01:59.724581+0000","last_clean":"2024-09-16T05:01:59.724581+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:01:59.724581+0000","last_undegraded":"2024-09-16T05:01:59.724581+0000","last_fullsized":"2024-09-16T05:01:59.724581+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3919,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36823,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166760448,"data_stored":166743053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493272,"kb_used_data":82564,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93878568,"statfs":{"total":96636764160,"available":96131653632,"internally_reserved":0,"allocated":84545536,"data_stored":84182314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593884,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501488,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870352,"statfs":{"total":96636764160,"available":96123240448,"internally_reserved":0,"allocated":557056,"data_stored":186876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487828,"kb_used_data":81340,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884012,"statfs":{"total":96636764160,"available":96137228288,"internally_reserved":0,"allocated":83292160,"data_stored":82928938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84000768,"data_stored":83997289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:40.559 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:40.940 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:40.959 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2024-09-16T05:02:40.613192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37412,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1418552,"kb_used_data":96252,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281696968,"statfs":{"total":289910292480,"available":288457695232,"internally_reserved":0,"allocated":98562048,"data_stored":97463888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1353960449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001893"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20407,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754061+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754061+0000","last_peered":"2024-09-16T05:02:39.754061+0000","last_clean":"2024-09-16T05:02:39.754061+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754061+0000","last_undegraded":"2024-09-16T05:02:39.754061+0000","last_fullsized":"2024-09-16T05:02:39.754061+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20746,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:40.167487+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:40.167487+0000","last_peered":"2024-09-16T05:02:40.167487+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:40.167487+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26869,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738925+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:02:39.738925+0000","last_peered":"2024-09-16T05:02:39.738925+0000","last_clean":"2024-09-16T05:02:39.738925+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:39.738925+0000","last_undegraded":"2024-09-16T05:02:39.738925+0000","last_fullsized":"2024-09-16T05:02:39.738925+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:41.505250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26879,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740115+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740115+0000","last_peered":"2024-09-16T05:02:39.740115+0000","last_clean":"2024-09-16T05:02:39.740115+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740115+0000","last_undegraded":"2024-09-16T05:02:39.740115+0000","last_fullsized":"2024-09-16T05:02:39.740115+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37411,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425068,"kb_used_data":14360,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946772,"statfs":{"total":96636764160,"available":96201494528,"internally_reserved":0,"allocated":14704640,"data_stored":14343624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593885,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505652,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866188,"statfs":{"total":96636764160,"available":96118976512,"internally_reserved":0,"allocated":557056,"data_stored":188484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487832,"kb_used_data":81348,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884008,"statfs":{"total":96636764160,"available":96137224192,"internally_reserved":0,"allocated":83300352,"data_stored":82931780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:40.960 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:41.298 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:41.298 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:41.317 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2024-09-16T05:02:40.613192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37412,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1418552,"kb_used_data":96252,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281696968,"statfs":{"total":289910292480,"available":288457695232,"internally_reserved":0,"allocated":98562048,"data_stored":97463888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1353960449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001893"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20407,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754061+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754061+0000","last_peered":"2024-09-16T05:02:39.754061+0000","last_clean":"2024-09-16T05:02:39.754061+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754061+0000","last_undegraded":"2024-09-16T05:02:39.754061+0000","last_fullsized":"2024-09-16T05:02:39.754061+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20746,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:40.167487+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:40.167487+0000","last_peered":"2024-09-16T05:02:40.167487+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:40.167487+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26869,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738925+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:02:39.738925+0000","last_peered":"2024-09-16T05:02:39.738925+0000","last_clean":"2024-09-16T05:02:39.738925+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:39.738925+0000","last_undegraded":"2024-09-16T05:02:39.738925+0000","last_fullsized":"2024-09-16T05:02:39.738925+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:41.505250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26879,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740115+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740115+0000","last_peered":"2024-09-16T05:02:39.740115+0000","last_clean":"2024-09-16T05:02:39.740115+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740115+0000","last_undegraded":"2024-09-16T05:02:39.740115+0000","last_fullsized":"2024-09-16T05:02:39.740115+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37411,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425068,"kb_used_data":14360,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946772,"statfs":{"total":96636764160,"available":96201494528,"internally_reserved":0,"allocated":14704640,"data_stored":14343624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593885,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505652,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866188,"statfs":{"total":96636764160,"available":96118976512,"internally_reserved":0,"allocated":557056,"data_stored":188484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487832,"kb_used_data":81348,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884008,"statfs":{"total":96636764160,"available":96137224192,"internally_reserved":0,"allocated":83300352,"data_stored":82931780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:41.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:41.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20746, 'reported_epoch': 67, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:40.167487+0000', 'last_change': '2024-09-16T05:02:38.437501+0000', 'last_active': '2024-09-16T05:02:40.167487+0000', 'last_peered': '2024-09-16T05:02:40.167487+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:02:40.167487+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3331, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 127, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3331}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:41.319 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:41.812 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:41.812 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3331/18 objects degraded (18505.556%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":909,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1452597248,"bytes_avail":288457695232,"bytes_total":289910292480,"degraded_objects":3331,"degraded_total":18,"degraded_ratio":185.05555555555554,"recovering_objects_per_sec":140,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1125,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:41.836 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:02:41.836 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:42.139 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:42.140 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:42.158 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2024-09-16T05:02:40.613192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37412,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1418552,"kb_used_data":96252,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281696968,"statfs":{"total":289910292480,"available":288457695232,"internally_reserved":0,"allocated":98562048,"data_stored":97463888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1353960449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001893"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20407,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754061+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754061+0000","last_peered":"2024-09-16T05:02:39.754061+0000","last_clean":"2024-09-16T05:02:39.754061+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754061+0000","last_undegraded":"2024-09-16T05:02:39.754061+0000","last_fullsized":"2024-09-16T05:02:39.754061+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20746,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:40.167487+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:40.167487+0000","last_peered":"2024-09-16T05:02:40.167487+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:40.167487+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26869,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738925+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:02:39.738925+0000","last_peered":"2024-09-16T05:02:39.738925+0000","last_clean":"2024-09-16T05:02:39.738925+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:39.738925+0000","last_undegraded":"2024-09-16T05:02:39.738925+0000","last_fullsized":"2024-09-16T05:02:39.738925+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:41.505250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26879,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740115+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740115+0000","last_peered":"2024-09-16T05:02:39.740115+0000","last_clean":"2024-09-16T05:02:39.740115+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740115+0000","last_undegraded":"2024-09-16T05:02:39.740115+0000","last_fullsized":"2024-09-16T05:02:39.740115+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37411,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425068,"kb_used_data":14360,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946772,"statfs":{"total":96636764160,"available":96201494528,"internally_reserved":0,"allocated":14704640,"data_stored":14343624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593885,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505652,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866188,"statfs":{"total":96636764160,"available":96118976512,"internally_reserved":0,"allocated":557056,"data_stored":188484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487832,"kb_used_data":81348,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884008,"statfs":{"total":96636764160,"available":96137224192,"internally_reserved":0,"allocated":83300352,"data_stored":82931780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:45.160 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:45.547 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:45.548 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:45.568 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-09-16T05:02:44.614169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37412,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1418552,"kb_used_data":96252,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281696968,"statfs":{"total":289910292480,"available":288457695232,"internally_reserved":0,"allocated":98562048,"data_stored":97463888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1353960449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":588,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001979"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20407,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754061+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754061+0000","last_peered":"2024-09-16T05:02:39.754061+0000","last_clean":"2024-09-16T05:02:39.754061+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754061+0000","last_undegraded":"2024-09-16T05:02:39.754061+0000","last_fullsized":"2024-09-16T05:02:39.754061+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20746,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:40.167487+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:40.167487+0000","last_peered":"2024-09-16T05:02:40.167487+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:40.167487+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26869,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738925+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:02:39.738925+0000","last_peered":"2024-09-16T05:02:39.738925+0000","last_clean":"2024-09-16T05:02:39.738925+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:39.738925+0000","last_undegraded":"2024-09-16T05:02:39.738925+0000","last_fullsized":"2024-09-16T05:02:39.738925+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:41.505250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26879,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740115+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740115+0000","last_peered":"2024-09-16T05:02:39.740115+0000","last_clean":"2024-09-16T05:02:39.740115+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740115+0000","last_undegraded":"2024-09-16T05:02:39.740115+0000","last_fullsized":"2024-09-16T05:02:39.740115+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37411,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425068,"kb_used_data":14360,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946772,"statfs":{"total":96636764160,"available":96201494528,"internally_reserved":0,"allocated":14704640,"data_stored":14343624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593885,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505652,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866188,"statfs":{"total":96636764160,"available":96118976512,"internally_reserved":0,"allocated":557056,"data_stored":188484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487832,"kb_used_data":81348,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884008,"statfs":{"total":96636764160,"available":96137224192,"internally_reserved":0,"allocated":83300352,"data_stored":82931780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:45.569 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:45.900 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:45.923 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-09-16T05:02:44.614169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286625,"num_write_kb":503699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37412,"num_bytes_recovered":31309843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1418552,"kb_used_data":96252,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281696968,"statfs":{"total":289910292480,"available":288457695232,"internally_reserved":0,"allocated":98562048,"data_stored":97463888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1353960449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":588,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001979"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20407,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754061+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754061+0000","last_peered":"2024-09-16T05:02:39.754061+0000","last_clean":"2024-09-16T05:02:39.754061+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754061+0000","last_undegraded":"2024-09-16T05:02:39.754061+0000","last_fullsized":"2024-09-16T05:02:39.754061+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20746,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:40.167487+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:40.167487+0000","last_peered":"2024-09-16T05:02:40.167487+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:40.167487+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26869,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738925+0000","last_change":"2024-09-16T04:56:42.830304+0000","last_active":"2024-09-16T05:02:39.738925+0000","last_peered":"2024-09-16T05:02:39.738925+0000","last_clean":"2024-09-16T05:02:39.738925+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:39.738925+0000","last_undegraded":"2024-09-16T05:02:39.738925+0000","last_fullsized":"2024-09-16T05:02:39.738925+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":3199,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:41.505250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19951,"num_write_kb":39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3300,"num_bytes_recovered":3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26879,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740115+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740115+0000","last_peered":"2024-09-16T05:02:39.740115+0000","last_clean":"2024-09-16T05:02:39.740115+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740115+0000","last_undegraded":"2024-09-16T05:02:39.740115+0000","last_fullsized":"2024-09-16T05:02:39.740115+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4319,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":286623,"num_write_kb":503697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37411,"num_bytes_recovered":31309824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425068,"kb_used_data":14360,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946772,"statfs":{"total":96636764160,"available":96201494528,"internally_reserved":0,"allocated":14704640,"data_stored":14343624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593885,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505652,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866188,"statfs":{"total":96636764160,"available":96118976512,"internally_reserved":0,"allocated":557056,"data_stored":188484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487832,"kb_used_data":81348,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884008,"statfs":{"total":96636764160,"available":96137224192,"internally_reserved":0,"allocated":83300352,"data_stored":82931780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:45.924 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:45.924 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 20746, 'reported_epoch': 67, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:40.167487+0000', 'last_change': '2024-09-16T05:02:38.437501+0000', 'last_active': '2024-09-16T05:02:40.167487+0000', 'last_peered': '2024-09-16T05:02:40.167487+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:02:40.167487+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3331, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 127, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3331}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:45.924 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:46.350 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:46.350 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3331/18 objects degraded (18505.556%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":914,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9,"data_bytes":4338,"bytes_used":1452597248,"bytes_avail":288457695232,"bytes_total":289910292480,"degraded_objects":3331,"degraded_total":18,"degraded_ratio":185.05555555555554,"recovering_objects_per_sec":73,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":588,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:46.370 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:02:46.371 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:46.725 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:46.726 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:46.746 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":519,"stamp":"2024-09-16T05:02:46.614724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34760,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1422736,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281692784,"statfs":{"total":289910292480,"available":288453410816,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1358220289},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1234,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-19951,"num_write_kb":-39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2064,"num_bytes_recovered":-3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002020"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20407,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754061+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754061+0000","last_peered":"2024-09-16T05:02:39.754061+0000","last_clean":"2024-09-16T05:02:39.754061+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754061+0000","last_undegraded":"2024-09-16T05:02:39.754061+0000","last_fullsized":"2024-09-16T05:02:39.754061+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":22042,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:45.163457+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:45.163457+0000","last_peered":"2024-09-16T05:02:45.163457+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:45.163457+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":775,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26879,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740115+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740115+0000","last_peered":"2024-09-16T05:02:39.740115+0000","last_clean":"2024-09-16T05:02:39.740115+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740115+0000","last_undegraded":"2024-09-16T05:02:39.740115+0000","last_fullsized":"2024-09-16T05:02:39.740115+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34759,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593886,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509816,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862024,"statfs":{"total":96636764160,"available":96114712576,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:49.749 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:50.110 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:50.132 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-09-16T05:02:48.615157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34760,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1422736,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281692784,"statfs":{"total":289910292480,"available":288453410816,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1358220289},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1234,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-19951,"num_write_kb":-39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2064,"num_bytes_recovered":-3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.893651"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20407,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754061+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754061+0000","last_peered":"2024-09-16T05:02:39.754061+0000","last_clean":"2024-09-16T05:02:39.754061+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754061+0000","last_undegraded":"2024-09-16T05:02:39.754061+0000","last_fullsized":"2024-09-16T05:02:39.754061+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":22042,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:45.163457+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:45.163457+0000","last_peered":"2024-09-16T05:02:45.163457+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:45.163457+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":775,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26879,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740115+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740115+0000","last_peered":"2024-09-16T05:02:39.740115+0000","last_clean":"2024-09-16T05:02:39.740115+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740115+0000","last_undegraded":"2024-09-16T05:02:39.740115+0000","last_fullsized":"2024-09-16T05:02:39.740115+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34759,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593886,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509816,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862024,"statfs":{"total":96636764160,"available":96114712576,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:50.134 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:50.493 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-09-16T05:02:48.615157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34760,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1422736,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281692784,"statfs":{"total":289910292480,"available":288453410816,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1358220289},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1234,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-19951,"num_write_kb":-39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2064,"num_bytes_recovered":-3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.893651"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20407,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754061+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754061+0000","last_peered":"2024-09-16T05:02:39.754061+0000","last_clean":"2024-09-16T05:02:39.754061+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754061+0000","last_undegraded":"2024-09-16T05:02:39.754061+0000","last_fullsized":"2024-09-16T05:02:39.754061+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":22042,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:45.163457+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:45.163457+0000","last_peered":"2024-09-16T05:02:45.163457+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:45.163457+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":775,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26879,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740115+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740115+0000","last_peered":"2024-09-16T05:02:39.740115+0000","last_clean":"2024-09-16T05:02:39.740115+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740115+0000","last_undegraded":"2024-09-16T05:02:39.740115+0000","last_fullsized":"2024-09-16T05:02:39.740115+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34759,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593886,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509816,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862024,"statfs":{"total":96636764160,"available":96114712576,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:50.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:50.514 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 22042, 'reported_epoch': 67, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:45.163457+0000', 'last_change': '2024-09-16T05:02:38.437501+0000', 'last_active': '2024-09-16T05:02:45.163457+0000', 'last_peered': '2024-09-16T05:02:45.163457+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:02:45.163457+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2683, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 775, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2683}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:50.514 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:50.885 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:50.885 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1922/14 objects degraded (13728.571%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":918,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":7,"data_bytes":3104,"bytes_used":1448361984,"bytes_avail":288461930496,"bytes_total":289910292480,"degraded_objects":1922,"degraded_total":14,"degraded_ratio":137.28571428571428},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:50.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:no progress seen, keeping timeout for now 2024-09-16T05:02:50.908 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:51.212 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:51.212 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:51.233 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":521,"stamp":"2024-09-16T05:02:50.615868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35521,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1414416,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1318067,"kb_avail":281701104,"statfs":{"total":289910292480,"available":288461930496,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1349700609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1234,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-19951,"num_write_kb":-39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1303,"num_bytes_recovered":-3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.880364"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20407,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754061+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754061+0000","last_peered":"2024-09-16T05:02:39.754061+0000","last_clean":"2024-09-16T05:02:39.754061+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754061+0000","last_undegraded":"2024-09-16T05:02:39.754061+0000","last_fullsized":"2024-09-16T05:02:39.754061+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":23564,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:50.167765+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:50.167765+0000","last_peered":"2024-09-16T05:02:50.167765+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:50.167765+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1536,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1922}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26880,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740256+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740256+0000","last_peered":"2024-09-16T05:02:39.740256+0000","last_clean":"2024-09-16T05:02:39.740256+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740256+0000","last_undegraded":"2024-09-16T05:02:39.740256+0000","last_fullsized":"2024-09-16T05:02:39.740256+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35520,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593887,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501496,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870344,"statfs":{"total":96636764160,"available":96123232256,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:54.235 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:54.602 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:54.603 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:54.624 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2024-09-16T05:02:52.616501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35521,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1414416,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1318067,"kb_avail":281701104,"statfs":{"total":289910292480,"available":288461930496,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1349700609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1234,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-19951,"num_write_kb":-39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1891,"num_bytes_recovered":-3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003309"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20407,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754061+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754061+0000","last_peered":"2024-09-16T05:02:39.754061+0000","last_clean":"2024-09-16T05:02:39.754061+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754061+0000","last_undegraded":"2024-09-16T05:02:39.754061+0000","last_fullsized":"2024-09-16T05:02:39.754061+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":23564,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:50.167765+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:50.167765+0000","last_peered":"2024-09-16T05:02:50.167765+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:50.167765+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1536,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1922}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26880,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740256+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740256+0000","last_peered":"2024-09-16T05:02:39.740256+0000","last_clean":"2024-09-16T05:02:39.740256+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740256+0000","last_undegraded":"2024-09-16T05:02:39.740256+0000","last_fullsized":"2024-09-16T05:02:39.740256+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35520,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593887,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501496,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870344,"statfs":{"total":96636764160,"available":96123232256,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:54.625 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:54.940 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:54.959 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2024-09-16T05:02:54.617029+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35521,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1414416,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1318067,"kb_avail":281701104,"statfs":{"total":289910292480,"available":288461930496,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1349700609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1234,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-19951,"num_write_kb":-39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1891,"num_bytes_recovered":-3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003332"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20407,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754061+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754061+0000","last_peered":"2024-09-16T05:02:39.754061+0000","last_clean":"2024-09-16T05:02:39.754061+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754061+0000","last_undegraded":"2024-09-16T05:02:39.754061+0000","last_fullsized":"2024-09-16T05:02:39.754061+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":23564,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:50.167765+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:50.167765+0000","last_peered":"2024-09-16T05:02:50.167765+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:50.167765+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1536,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1922}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26880,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740256+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740256+0000","last_peered":"2024-09-16T05:02:39.740256+0000","last_clean":"2024-09-16T05:02:39.740256+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740256+0000","last_undegraded":"2024-09-16T05:02:39.740256+0000","last_fullsized":"2024-09-16T05:02:39.740256+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35520,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593887,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501496,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870344,"statfs":{"total":96636764160,"available":96123232256,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:54.960 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:54.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 23564, 'reported_epoch': 67, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:50.167765+0000', 'last_change': '2024-09-16T05:02:38.437501+0000', 'last_active': '2024-09-16T05:02:50.167765+0000', 'last_peered': '2024-09-16T05:02:50.167765+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:02:50.167765+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1922, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1536, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1922}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:54.961 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:55.357 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:55.357 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1922/14 objects degraded (13728.571%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":923,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":7,"data_bytes":3104,"bytes_used":1448361984,"bytes_avail":288461930496,"bytes_total":289910292480,"degraded_objects":1922,"degraded_total":14,"degraded_ratio":137.28571428571428},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:55.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:no progress seen, keeping timeout for now 2024-09-16T05:02:55.374 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:55.685 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:55.704 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2024-09-16T05:02:54.617029+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35521,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1414416,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1318067,"kb_avail":281701104,"statfs":{"total":289910292480,"available":288461930496,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1349700609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1234,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-19951,"num_write_kb":-39899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1891,"num_bytes_recovered":-3981312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003332"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20407,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754061+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754061+0000","last_peered":"2024-09-16T05:02:39.754061+0000","last_clean":"2024-09-16T05:02:39.754061+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754061+0000","last_undegraded":"2024-09-16T05:02:39.754061+0000","last_fullsized":"2024-09-16T05:02:39.754061+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":23564,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:50.167765+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:50.167765+0000","last_peered":"2024-09-16T05:02:50.167765+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:50.167765+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1536,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1922}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26880,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740256+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740256+0000","last_peered":"2024-09-16T05:02:39.740256+0000","last_clean":"2024-09-16T05:02:39.740256+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740256+0000","last_undegraded":"2024-09-16T05:02:39.740256+0000","last_fullsized":"2024-09-16T05:02:39.740256+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1922,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35520,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593887,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501496,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870344,"statfs":{"total":96636764160,"available":96123232256,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:58.707 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:59.062 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:59.079 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2024-09-16T05:02:58.618003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1460,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35983,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1418576,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281696944,"statfs":{"total":289910292480,"available":288457670656,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1353960449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1223,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003279"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20408,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754123+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754123+0000","last_peered":"2024-09-16T05:02:39.754123+0000","last_clean":"2024-09-16T05:02:39.754123+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754123+0000","last_undegraded":"2024-09-16T05:02:39.754123+0000","last_fullsized":"2024-09-16T05:02:39.754123+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":24488,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:55.171072+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:55.171072+0000","last_peered":"2024-09-16T05:02:55.171072+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:55.171072+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1460,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1998,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1460}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26880,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740256+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740256+0000","last_peered":"2024-09-16T05:02:39.740256+0000","last_clean":"2024-09-16T05:02:39.740256+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740256+0000","last_undegraded":"2024-09-16T05:02:39.740256+0000","last_fullsized":"2024-09-16T05:02:39.740256+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1460,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35982,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593888,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505656,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866184,"statfs":{"total":96636764160,"available":96118972416,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:59.081 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:02:59.435 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:02:59.454 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2024-09-16T05:02:58.618003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1460,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35983,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1418576,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281696944,"statfs":{"total":289910292480,"available":288457670656,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1353960449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1223,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003279"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20408,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754123+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754123+0000","last_peered":"2024-09-16T05:02:39.754123+0000","last_clean":"2024-09-16T05:02:39.754123+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754123+0000","last_undegraded":"2024-09-16T05:02:39.754123+0000","last_fullsized":"2024-09-16T05:02:39.754123+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":24488,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:55.171072+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:55.171072+0000","last_peered":"2024-09-16T05:02:55.171072+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:55.171072+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1460,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1998,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1460}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26880,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740256+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740256+0000","last_peered":"2024-09-16T05:02:39.740256+0000","last_clean":"2024-09-16T05:02:39.740256+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740256+0000","last_undegraded":"2024-09-16T05:02:39.740256+0000","last_fullsized":"2024-09-16T05:02:39.740256+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1460,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35982,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593888,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505656,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866184,"statfs":{"total":96636764160,"available":96118972416,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:02:59.455 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:02:59.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 24488, 'reported_epoch': 67, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:02:55.171072+0000', 'last_change': '2024-09-16T05:02:38.437501+0000', 'last_active': '2024-09-16T05:02:55.171072+0000', 'last_peered': '2024-09-16T05:02:55.171072+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:02:55.171072+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1460, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1998, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1460}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:02:59.456 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:02:59.840 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:02:59.840 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1460/14 objects degraded (10428.571%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":927,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":7,"data_bytes":3104,"bytes_used":1452621824,"bytes_avail":288457670656,"bytes_total":289910292480,"degraded_objects":1460,"degraded_total":14,"degraded_ratio":104.28571428571429,"recovering_objects_per_sec":101,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1223,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:02:59.864 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:02:59.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:00.179 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:00.180 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:00.199 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2024-09-16T05:02:58.618003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1460,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35983,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1418576,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281696944,"statfs":{"total":289910292480,"available":288457670656,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1353960449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1223,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003279"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20408,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754123+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754123+0000","last_peered":"2024-09-16T05:02:39.754123+0000","last_clean":"2024-09-16T05:02:39.754123+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754123+0000","last_undegraded":"2024-09-16T05:02:39.754123+0000","last_fullsized":"2024-09-16T05:02:39.754123+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":24488,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:02:55.171072+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:02:55.171072+0000","last_peered":"2024-09-16T05:02:55.171072+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:02:55.171072+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1460,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1998,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1460}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":143,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753810+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753810+0000","last_peered":"2024-09-16T05:02:39.753810+0000","last_clean":"2024-09-16T05:02:39.753810+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753810+0000","last_undegraded":"2024-09-16T05:02:39.753810+0000","last_fullsized":"2024-09-16T05:02:39.753810+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26880,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740256+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740256+0000","last_peered":"2024-09-16T05:02:39.740256+0000","last_clean":"2024-09-16T05:02:39.740256+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740256+0000","last_undegraded":"2024-09-16T05:02:39.740256+0000","last_fullsized":"2024-09-16T05:02:39.740256+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1460,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35982,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593888,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505656,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866184,"statfs":{"total":96636764160,"available":96118972416,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":517205678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:03.201 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:03.568 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:03.588 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":527,"stamp":"2024-09-16T05:03:02.619175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36456,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1422736,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281692784,"statfs":{"total":289910292480,"available":288453410816,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1358220289},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":935,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003306"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20408,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754123+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754123+0000","last_peered":"2024-09-16T05:02:39.754123+0000","last_clean":"2024-09-16T05:02:39.754123+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754123+0000","last_undegraded":"2024-09-16T05:02:39.754123+0000","last_fullsized":"2024-09-16T05:02:39.754123+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":25434,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:03:00.168669+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:03:00.168669+0000","last_peered":"2024-09-16T05:03:00.168669+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:03:00.168669+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2471,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753896+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753896+0000","last_peered":"2024-09-16T05:02:39.753896+0000","last_clean":"2024-09-16T05:02:39.753896+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753896+0000","last_undegraded":"2024-09-16T05:02:39.753896+0000","last_fullsized":"2024-09-16T05:02:39.753896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26880,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740256+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740256+0000","last_peered":"2024-09-16T05:02:39.740256+0000","last_clean":"2024-09-16T05:02:39.740256+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740256+0000","last_undegraded":"2024-09-16T05:02:39.740256+0000","last_fullsized":"2024-09-16T05:02:39.740256+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36455,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593889,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509816,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862024,"statfs":{"total":96636764160,"available":96114712576,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:03.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:03.908 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:03.909 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:03.928 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":527,"stamp":"2024-09-16T05:03:02.619175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36456,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1422736,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281692784,"statfs":{"total":289910292480,"available":288453410816,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1358220289},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":935,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003306"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20408,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754123+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754123+0000","last_peered":"2024-09-16T05:02:39.754123+0000","last_clean":"2024-09-16T05:02:39.754123+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754123+0000","last_undegraded":"2024-09-16T05:02:39.754123+0000","last_fullsized":"2024-09-16T05:02:39.754123+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":25434,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:03:00.168669+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:03:00.168669+0000","last_peered":"2024-09-16T05:03:00.168669+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:03:00.168669+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2471,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753896+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753896+0000","last_peered":"2024-09-16T05:02:39.753896+0000","last_clean":"2024-09-16T05:02:39.753896+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753896+0000","last_undegraded":"2024-09-16T05:02:39.753896+0000","last_fullsized":"2024-09-16T05:02:39.753896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26880,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740256+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740256+0000","last_peered":"2024-09-16T05:02:39.740256+0000","last_clean":"2024-09-16T05:02:39.740256+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740256+0000","last_undegraded":"2024-09-16T05:02:39.740256+0000","last_fullsized":"2024-09-16T05:02:39.740256+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36455,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593889,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509816,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862024,"statfs":{"total":96636764160,"available":96114712576,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:03.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:03:03.930 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 25434, 'reported_epoch': 67, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:03:00.168669+0000', 'last_change': '2024-09-16T05:02:38.437501+0000', 'last_active': '2024-09-16T05:03:00.168669+0000', 'last_peered': '2024-09-16T05:03:00.168669+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:03:00.168669+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 987, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2471, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -987}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:03:03.930 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:03:04.308 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:04.308 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 987/14 objects degraded (7050.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":932,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":7,"data_bytes":3104,"bytes_used":1456881664,"bytes_avail":288453410816,"bytes_total":289910292480,"degraded_objects":987,"degraded_total":14,"degraded_ratio":70.5,"recovering_objects_per_sec":77,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":935,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:03:04.328 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:03:04.329 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:04.640 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:04.641 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:04.658 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":528,"stamp":"2024-09-16T05:03:04.619836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36456,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1422736,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281692784,"statfs":{"total":289910292480,"available":288453410816,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1358220289},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":935,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003335"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20408,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754123+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754123+0000","last_peered":"2024-09-16T05:02:39.754123+0000","last_clean":"2024-09-16T05:02:39.754123+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754123+0000","last_undegraded":"2024-09-16T05:02:39.754123+0000","last_fullsized":"2024-09-16T05:02:39.754123+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":25434,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:03:00.168669+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:03:00.168669+0000","last_peered":"2024-09-16T05:03:00.168669+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:03:00.168669+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2471,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753896+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753896+0000","last_peered":"2024-09-16T05:02:39.753896+0000","last_clean":"2024-09-16T05:02:39.753896+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753896+0000","last_undegraded":"2024-09-16T05:02:39.753896+0000","last_fullsized":"2024-09-16T05:02:39.753896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26880,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740256+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740256+0000","last_peered":"2024-09-16T05:02:39.740256+0000","last_clean":"2024-09-16T05:02:39.740256+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740256+0000","last_undegraded":"2024-09-16T05:02:39.740256+0000","last_fullsized":"2024-09-16T05:02:39.740256+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36455,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593889,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509816,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862024,"statfs":{"total":96636764160,"available":96114712576,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":521465518},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:07.661 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:07.980 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:07.999 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":529,"stamp":"2024-09-16T05:03:06.620147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37009,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1410256,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1313907,"kb_avail":281705264,"statfs":{"total":289910292480,"available":288466190336,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1345440769},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1488,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003117"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20408,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754123+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754123+0000","last_peered":"2024-09-16T05:02:39.754123+0000","last_clean":"2024-09-16T05:02:39.754123+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754123+0000","last_undegraded":"2024-09-16T05:02:39.754123+0000","last_fullsized":"2024-09-16T05:02:39.754123+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":26540,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:03:05.167649+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:03:05.167649+0000","last_peered":"2024-09-16T05:03:05.167649+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:03:05.167649+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3024,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-434}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753896+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753896+0000","last_peered":"2024-09-16T05:02:39.753896+0000","last_clean":"2024-09-16T05:02:39.753896+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753896+0000","last_undegraded":"2024-09-16T05:02:39.753896+0000","last_fullsized":"2024-09-16T05:02:39.753896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26880,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740256+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740256+0000","last_peered":"2024-09-16T05:02:39.740256+0000","last_clean":"2024-09-16T05:02:39.740256+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740256+0000","last_undegraded":"2024-09-16T05:02:39.740256+0000","last_fullsized":"2024-09-16T05:02:39.740256+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37008,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593890,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497336,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874504,"statfs":{"total":96636764160,"available":96127492096,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182581,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:08.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:08.302 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:08.319 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":529,"stamp":"2024-09-16T05:03:06.620147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37009,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1410256,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1313907,"kb_avail":281705264,"statfs":{"total":289910292480,"available":288466190336,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1345440769},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1488,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1488,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003117"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20408,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754123+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754123+0000","last_peered":"2024-09-16T05:02:39.754123+0000","last_clean":"2024-09-16T05:02:39.754123+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754123+0000","last_undegraded":"2024-09-16T05:02:39.754123+0000","last_fullsized":"2024-09-16T05:02:39.754123+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":26540,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:03:05.167649+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:03:05.167649+0000","last_peered":"2024-09-16T05:03:05.167649+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:03:05.167649+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3024,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-434}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753896+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753896+0000","last_peered":"2024-09-16T05:02:39.753896+0000","last_clean":"2024-09-16T05:02:39.753896+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753896+0000","last_undegraded":"2024-09-16T05:02:39.753896+0000","last_fullsized":"2024-09-16T05:02:39.753896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26880,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740256+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740256+0000","last_peered":"2024-09-16T05:02:39.740256+0000","last_clean":"2024-09-16T05:02:39.740256+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740256+0000","last_undegraded":"2024-09-16T05:02:39.740256+0000","last_fullsized":"2024-09-16T05:02:39.740256+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37008,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593890,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497336,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874504,"statfs":{"total":96636764160,"available":96127492096,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182581,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:08.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-16T05:03:08.321 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "51'20313", 'reported_seq': 26540, 'reported_epoch': 67, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-16T05:03:05.167649+0000', 'last_change': '2024-09-16T05:02:38.437501+0000', 'last_active': '2024-09-16T05:03:05.167649+0000', 'last_peered': '2024-09-16T05:03:05.167649+0000', 'last_clean': '2024-09-16T04:49:35.646925+0000', 'last_became_active': '2024-09-16T04:49:38.292037+0000', 'last_became_peered': '2024-09-16T04:49:38.292037+0000', 'last_unstale': '2024-09-16T05:03:05.167649+0000', 'last_undegraded': '2024-09-16T04:49:37.759077+0000', 'last_fullsized': '2024-09-16T04:49:37.653552+0000', 'mapping_epoch': 44, 'log_start': "47'10300", 'ondisk_log_start': "47'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3046", 'last_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T04:47:55.367805+0000', 'last_clean_scrub_stamp': '2024-09-16T04:49:25.570345+0000', 'objects_scrubbed': 3002, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1339, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 434, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3024, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -434}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T05:03:08.321 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T05:03:08.691 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:08.692 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 434/14 objects degraded (3100.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":936,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":68,"num_osds":3,"num_up_osds":3,"osd_up_since":1726462175,"num_in_osds":3,"osd_in_since":1726462168,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":7,"data_bytes":3104,"bytes_used":1444102144,"bytes_avail":288466190336,"bytes_total":289910292480,"degraded_objects":434,"degraded_total":14,"degraded_ratio":31,"recovering_objects_per_sec":85,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1026,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-16T04:47:27:159526+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T04:50:36.296143+0000","services":{}},"progress_events":{}} 2024-09-16T05:03:08.712 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-16T05:03:08.712 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:09.005 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:09.022 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":531,"stamp":"2024-09-16T05:03:08.685005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3104,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266674,"num_write_kb":463800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37009,"num_bytes_recovered":27328531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1410256,"kb_used_data":96276,"kb_used_omap":12,"kb_used_meta":1313907,"kb_avail":281705264,"statfs":{"total":289910292480,"available":288466190336,"internally_reserved":0,"allocated":98586624,"data_stored":97472351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1345440769},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.067002"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26551,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754463+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:02:39.754463+0000","last_peered":"2024-09-16T05:02:39.754463+0000","last_clean":"2024-09-16T05:02:39.754463+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:02:39.754463+0000","last_undegraded":"2024-09-16T05:02:39.754463+0000","last_fullsized":"2024-09-16T05:02:39.754463+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20155,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.739792+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:02:39.739792+0000","last_peered":"2024-09-16T05:02:39.739792+0000","last_clean":"2024-09-16T05:02:39.739792+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:02:39.739792+0000","last_undegraded":"2024-09-16T05:02:39.739792+0000","last_fullsized":"2024-09-16T05:02:39.739792+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20020,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.931149+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:02:39.931149+0000","last_peered":"2024-09-16T05:02:39.931149+0000","last_clean":"2024-09-16T05:02:39.931149+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:02:39.931149+0000","last_undegraded":"2024-09-16T05:02:39.931149+0000","last_fullsized":"2024-09-16T05:02:39.931149+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20459,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738422+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:02:39.738422+0000","last_peered":"2024-09-16T05:02:39.738422+0000","last_clean":"2024-09-16T05:02:39.738422+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:02:39.738422+0000","last_undegraded":"2024-09-16T05:02:39.738422+0000","last_fullsized":"2024-09-16T05:02:39.738422+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20408,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.754123+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:02:39.754123+0000","last_peered":"2024-09-16T05:02:39.754123+0000","last_clean":"2024-09-16T05:02:39.754123+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:02:39.754123+0000","last_undegraded":"2024-09-16T05:02:39.754123+0000","last_fullsized":"2024-09-16T05:02:39.754123+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":26540,"reported_epoch":67,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T05:03:05.167649+0000","last_change":"2024-09-16T05:02:38.437501+0000","last_active":"2024-09-16T05:03:05.167649+0000","last_peered":"2024-09-16T05:03:05.167649+0000","last_clean":"2024-09-16T04:49:35.646925+0000","last_became_active":"2024-09-16T04:49:38.292037+0000","last_became_peered":"2024-09-16T04:49:38.292037+0000","last_unstale":"2024-09-16T05:03:05.167649+0000","last_undegraded":"2024-09-16T04:49:37.759077+0000","last_fullsized":"2024-09-16T04:49:37.653552+0000","mapping_epoch":44,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3024,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-434}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20032,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741576+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:02:39.741576+0000","last_peered":"2024-09-16T05:02:39.741576+0000","last_clean":"2024-09-16T05:02:39.741576+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:02:39.741576+0000","last_undegraded":"2024-09-16T05:02:39.741576+0000","last_fullsized":"2024-09-16T05:02:39.741576+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20277,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753696+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:02:39.753696+0000","last_peered":"2024-09-16T05:02:39.753696+0000","last_clean":"2024-09-16T05:02:39.753696+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:02:39.753696+0000","last_undegraded":"2024-09-16T05:02:39.753696+0000","last_fullsized":"2024-09-16T05:02:39.753696+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.741697+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:02:39.741697+0000","last_peered":"2024-09-16T05:02:39.741697+0000","last_clean":"2024-09-16T05:02:39.741697+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:02:39.741697+0000","last_undegraded":"2024-09-16T05:02:39.741697+0000","last_fullsized":"2024-09-16T05:02:39.741697+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20340,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742194+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:02:39.742194+0000","last_peered":"2024-09-16T05:02:39.742194+0000","last_clean":"2024-09-16T05:02:39.742194+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:02:39.742194+0000","last_undegraded":"2024-09-16T05:02:39.742194+0000","last_fullsized":"2024-09-16T05:02:39.742194+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738644+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:02:39.738644+0000","last_peered":"2024-09-16T05:02:39.738644+0000","last_clean":"2024-09-16T05:02:39.738644+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:02:39.738644+0000","last_undegraded":"2024-09-16T05:02:39.738644+0000","last_fullsized":"2024-09-16T05:02:39.738644+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753991+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:02:39.753991+0000","last_peered":"2024-09-16T05:02:39.753991+0000","last_clean":"2024-09-16T05:02:39.753991+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:02:39.753991+0000","last_undegraded":"2024-09-16T05:02:39.753991+0000","last_fullsized":"2024-09-16T05:02:39.753991+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.753896+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:02:39.753896+0000","last_peered":"2024-09-16T05:02:39.753896+0000","last_clean":"2024-09-16T05:02:39.753896+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:02:39.753896+0000","last_undegraded":"2024-09-16T05:02:39.753896+0000","last_fullsized":"2024-09-16T05:02:39.753896+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26877,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:41.556511+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:02:41.556511+0000","last_peered":"2024-09-16T05:02:41.556511+0000","last_clean":"2024-09-16T05:02:41.556511+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:02:41.556511+0000","last_undegraded":"2024-09-16T05:02:41.556511+0000","last_fullsized":"2024-09-16T05:02:41.556511+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742043+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:02:39.742043+0000","last_peered":"2024-09-16T05:02:39.742043+0000","last_clean":"2024-09-16T05:02:39.742043+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:02:39.742043+0000","last_undegraded":"2024-09-16T05:02:39.742043+0000","last_fullsized":"2024-09-16T05:02:39.742043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26880,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.740256+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:02:39.740256+0000","last_peered":"2024-09-16T05:02:39.740256+0000","last_clean":"2024-09-16T05:02:39.740256+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:02:39.740256+0000","last_undegraded":"2024-09-16T05:02:39.740256+0000","last_fullsized":"2024-09-16T05:02:39.740256+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20013,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742756+0000","last_change":"2024-09-16T05:00:26.797007+0000","last_active":"2024-09-16T05:02:39.742756+0000","last_peered":"2024-09-16T05:02:39.742756+0000","last_clean":"2024-09-16T05:02:39.742756+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:02:39.742756+0000","last_undegraded":"2024-09-16T05:02:39.742756+0000","last_fullsized":"2024-09-16T05:02:39.742756+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:40.153029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19875,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3410,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742483+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:02:39.742483+0000","last_peered":"2024-09-16T05:02:39.742483+0000","last_clean":"2024-09-16T05:02:39.742483+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:02:39.742483+0000","last_undegraded":"2024-09-16T05:02:39.742483+0000","last_fullsized":"2024-09-16T05:02:39.742483+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20683,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742032+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:02:39.742032+0000","last_peered":"2024-09-16T05:02:39.742032+0000","last_clean":"2024-09-16T05:02:39.742032+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:02:39.742032+0000","last_undegraded":"2024-09-16T05:02:39.742032+0000","last_fullsized":"2024-09-16T05:02:39.742032+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:42.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":116,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.738848+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:02:39.738848+0000","last_peered":"2024-09-16T05:02:39.738848+0000","last_clean":"2024-09-16T05:02:39.738848+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:02:39.738848+0000","last_undegraded":"2024-09-16T05:02:39.738848+0000","last_fullsized":"2024-09-16T05:02:39.738848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27037,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742005+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:02:39.742005+0000","last_peered":"2024-09-16T05:02:39.742005+0000","last_clean":"2024-09-16T05:02:39.742005+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:02:39.742005+0000","last_undegraded":"2024-09-16T05:02:39.742005+0000","last_fullsized":"2024-09-16T05:02:39.742005+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742000+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:02:39.742000+0000","last_peered":"2024-09-16T05:02:39.742000+0000","last_clean":"2024-09-16T05:02:39.742000+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:02:39.742000+0000","last_undegraded":"2024-09-16T05:02:39.742000+0000","last_fullsized":"2024-09-16T05:02:39.742000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742675+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:02:39.742675+0000","last_peered":"2024-09-16T05:02:39.742675+0000","last_clean":"2024-09-16T05:02:39.742675+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:02:39.742675+0000","last_undegraded":"2024-09-16T05:02:39.742675+0000","last_fullsized":"2024-09-16T05:02:39.742675+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20598,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-16T05:02:39.742327+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:02:39.742327+0000","last_peered":"2024-09-16T05:02:39.742327+0000","last_clean":"2024-09-16T05:02:39.742327+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:02:39.742327+0000","last_undegraded":"2024-09-16T05:02:39.742327+0000","last_fullsized":"2024-09-16T05:02:39.742327+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3085,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":266672,"num_write_kb":463798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37008,"num_bytes_recovered":27328512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":96907264,"data_stored":96900306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425080,"kb_used_data":14368,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946760,"statfs":{"total":96636764160,"available":96201482240,"internally_reserved":0,"allocated":14712832,"data_stored":14346445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593890,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497336,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874504,"statfs":{"total":96636764160,"available":96127492096,"internally_reserved":0,"allocated":565248,"data_stored":191305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":508685998},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182581,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487840,"kb_used_data":81356,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884000,"statfs":{"total":96636764160,"available":96137216000,"internally_reserved":0,"allocated":83308544,"data_stored":82934601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82747392,"data_stored":82743913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14155776,"data_stored":14155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:12.024 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:12.336 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:12.337 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:12.354 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":533,"stamp":"2024-09-16T05:03:10.621171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2487,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246799,"num_write_kb":424051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34033,"num_bytes_recovered":27324435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413744,"kb_used_data":95604,"kb_used_omap":12,"kb_used_meta":1318067,"kb_avail":281701776,"statfs":{"total":289910292480,"available":288462618624,"internally_reserved":0,"allocated":97898496,"data_stored":96768098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1349700609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-617,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-19875,"num_write_kb":-39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2423,"num_bytes_recovered":-4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001996"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26555,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.695100+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:03:09.695100+0000","last_peered":"2024-09-16T05:03:09.695100+0000","last_clean":"2024-09-16T05:03:09.695100+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:03:09.695100+0000","last_undegraded":"2024-09-16T05:03:09.695100+0000","last_fullsized":"2024-09-16T05:03:09.695100+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20159,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053868+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:03:10.053868+0000","last_peered":"2024-09-16T05:03:10.053868+0000","last_clean":"2024-09-16T05:03:10.053868+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:03:10.053868+0000","last_undegraded":"2024-09-16T05:03:10.053868+0000","last_fullsized":"2024-09-16T05:03:10.053868+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20024,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692186+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:03:09.692186+0000","last_peered":"2024-09-16T05:03:09.692186+0000","last_clean":"2024-09-16T05:03:09.692186+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:03:09.692186+0000","last_undegraded":"2024-09-16T05:03:09.692186+0000","last_fullsized":"2024-09-16T05:03:09.692186+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20463,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053606+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:03:10.053606+0000","last_peered":"2024-09-16T05:03:10.053606+0000","last_clean":"2024-09-16T05:03:10.053606+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:03:10.053606+0000","last_undegraded":"2024-09-16T05:03:10.053606+0000","last_fullsized":"2024-09-16T05:03:10.053606+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20412,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.694074+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:03:09.694074+0000","last_peered":"2024-09-16T05:03:09.694074+0000","last_clean":"2024-09-16T05:03:09.694074+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:03:09.694074+0000","last_undegraded":"2024-09-16T05:03:09.694074+0000","last_fullsized":"2024-09-16T05:03:09.694074+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20467,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.871337+0000","last_change":"2024-09-16T05:03:09.870297+0000","last_active":"2024-09-16T05:03:09.871337+0000","last_peered":"2024-09-16T05:03:09.871337+0000","last_clean":"2024-09-16T05:03:09.871337+0000","last_became_active":"2024-09-16T05:03:09.695598+0000","last_became_peered":"2024-09-16T05:03:09.695598+0000","last_unstale":"2024-09-16T05:03:09.871337+0000","last_undegraded":"2024-09-16T05:03:09.871337+0000","last_fullsized":"2024-09-16T05:03:09.871337+0000","mapping_epoch":68,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:37.304390+0000","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691796+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:03:09.691796+0000","last_peered":"2024-09-16T05:03:09.691796+0000","last_clean":"2024-09-16T05:03:09.691796+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:03:09.691796+0000","last_undegraded":"2024-09-16T05:03:09.691796+0000","last_fullsized":"2024-09-16T05:03:09.691796+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20281,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693065+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:03:09.693065+0000","last_peered":"2024-09-16T05:03:09.693065+0000","last_clean":"2024-09-16T05:03:09.693065+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:03:09.693065+0000","last_undegraded":"2024-09-16T05:03:09.693065+0000","last_fullsized":"2024-09-16T05:03:09.693065+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691472+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:03:09.691472+0000","last_peered":"2024-09-16T05:03:09.691472+0000","last_clean":"2024-09-16T05:03:09.691472+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:03:09.691472+0000","last_undegraded":"2024-09-16T05:03:09.691472+0000","last_fullsized":"2024-09-16T05:03:09.691472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20344,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692824+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:03:09.692824+0000","last_peered":"2024-09-16T05:03:09.692824+0000","last_clean":"2024-09-16T05:03:09.692824+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:03:09.692824+0000","last_undegraded":"2024-09-16T05:03:09.692824+0000","last_fullsized":"2024-09-16T05:03:09.692824+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053275+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:03:10.053275+0000","last_peered":"2024-09-16T05:03:10.053275+0000","last_clean":"2024-09-16T05:03:10.053275+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:03:10.053275+0000","last_undegraded":"2024-09-16T05:03:10.053275+0000","last_fullsized":"2024-09-16T05:03:10.053275+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27338,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693866+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:03:09.693866+0000","last_peered":"2024-09-16T05:03:09.693866+0000","last_clean":"2024-09-16T05:03:09.693866+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:03:09.693866+0000","last_undegraded":"2024-09-16T05:03:09.693866+0000","last_fullsized":"2024-09-16T05:03:09.693866+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693578+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:03:09.693578+0000","last_peered":"2024-09-16T05:03:09.693578+0000","last_clean":"2024-09-16T05:03:09.693578+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:03:09.693578+0000","last_undegraded":"2024-09-16T05:03:09.693578+0000","last_fullsized":"2024-09-16T05:03:09.693578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26881,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053400+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:03:10.053400+0000","last_peered":"2024-09-16T05:03:10.053400+0000","last_clean":"2024-09-16T05:03:10.053400+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:03:10.053400+0000","last_undegraded":"2024-09-16T05:03:10.053400+0000","last_fullsized":"2024-09-16T05:03:10.053400+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692638+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:03:09.692638+0000","last_peered":"2024-09-16T05:03:09.692638+0000","last_clean":"2024-09-16T05:03:09.692638+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:03:09.692638+0000","last_undegraded":"2024-09-16T05:03:09.692638+0000","last_fullsized":"2024-09-16T05:03:09.692638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26884,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053718+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:03:10.053718+0000","last_peered":"2024-09-16T05:03:10.053718+0000","last_clean":"2024-09-16T05:03:10.053718+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:03:10.053718+0000","last_undegraded":"2024-09-16T05:03:10.053718+0000","last_fullsized":"2024-09-16T05:03:10.053718+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20025,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692648+0000","last_change":"2024-09-16T05:03:09.081130+0000","last_active":"2024-09-16T05:03:09.692648+0000","last_peered":"2024-09-16T05:03:09.692648+0000","last_clean":"2024-09-16T05:03:09.692648+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:03:09.692648+0000","last_undegraded":"2024-09-16T05:03:09.692648+0000","last_fullsized":"2024-09-16T05:03:09.692648+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19876","last_scrub_stamp":"2024-09-16T05:03:09.080970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:03:09.080970+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:19.155082+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693090+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:03:09.693090+0000","last_peered":"2024-09-16T05:03:09.693090+0000","last_clean":"2024-09-16T05:03:09.693090+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:03:09.693090+0000","last_undegraded":"2024-09-16T05:03:09.693090+0000","last_fullsized":"2024-09-16T05:03:09.693090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20687,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692091+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:03:09.692091+0000","last_peered":"2024-09-16T05:03:09.692091+0000","last_clean":"2024-09-16T05:03:09.692091+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:03:09.692091+0000","last_undegraded":"2024-09-16T05:03:09.692091+0000","last_fullsized":"2024-09-16T05:03:09.692091+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:03:10.053647+0000","last_peered":"2024-09-16T05:03:10.053647+0000","last_clean":"2024-09-16T05:03:10.053647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:03:10.053647+0000","last_undegraded":"2024-09-16T05:03:10.053647+0000","last_fullsized":"2024-09-16T05:03:10.053647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27041,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692471+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:03:09.692471+0000","last_peered":"2024-09-16T05:03:09.692471+0000","last_clean":"2024-09-16T05:03:09.692471+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:03:09.692471+0000","last_undegraded":"2024-09-16T05:03:09.692471+0000","last_fullsized":"2024-09-16T05:03:09.692471+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691799+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:03:09.691799+0000","last_peered":"2024-09-16T05:03:09.691799+0000","last_clean":"2024-09-16T05:03:09.691799+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:03:09.691799+0000","last_undegraded":"2024-09-16T05:03:09.691799+0000","last_fullsized":"2024-09-16T05:03:09.691799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692344+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:03:09.692344+0000","last_peered":"2024-09-16T05:03:09.692344+0000","last_clean":"2024-09-16T05:03:09.692344+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:03:09.692344+0000","last_undegraded":"2024-09-16T05:03:09.692344+0000","last_fullsized":"2024-09-16T05:03:09.692344+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20602,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693254+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:03:09.693254+0000","last_peered":"2024-09-16T05:03:09.693254+0000","last_clean":"2024-09-16T05:03:09.693254+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:03:09.693254+0000","last_undegraded":"2024-09-16T05:03:09.693254+0000","last_fullsized":"2024-09-16T05:03:09.693254+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246797,"num_write_kb":424049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34032,"num_bytes_recovered":27324416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":94232576,"data_stored":94225618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425084,"kb_used_data":14376,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946756,"statfs":{"total":96636764160,"available":96201478144,"internally_reserved":0,"allocated":14721024,"data_stored":14349262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593891,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501508,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870332,"statfs":{"total":96636764160,"available":96123219968,"internally_reserved":0,"allocated":573440,"data_stored":194122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487152,"kb_used_data":80668,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884688,"statfs":{"total":96636764160,"available":96137920512,"internally_reserved":0,"allocated":82604032,"data_stored":82224714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":81297408,"data_stored":81293929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12931072,"data_stored":12931072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:12.356 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-16T05:03:12.357 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-16T05:03:12.357 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:12.663 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:12.681 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2024-09-16T05:03:12.621690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2487,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246799,"num_write_kb":424051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34033,"num_bytes_recovered":27324435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413744,"kb_used_data":95604,"kb_used_omap":12,"kb_used_meta":1318067,"kb_avail":281701776,"statfs":{"total":289910292480,"available":288462618624,"internally_reserved":0,"allocated":97898496,"data_stored":96768098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1349700609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-617,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-19875,"num_write_kb":-39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2423,"num_bytes_recovered":-4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001854"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26555,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.695100+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:03:09.695100+0000","last_peered":"2024-09-16T05:03:09.695100+0000","last_clean":"2024-09-16T05:03:09.695100+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:03:09.695100+0000","last_undegraded":"2024-09-16T05:03:09.695100+0000","last_fullsized":"2024-09-16T05:03:09.695100+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20159,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053868+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:03:10.053868+0000","last_peered":"2024-09-16T05:03:10.053868+0000","last_clean":"2024-09-16T05:03:10.053868+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:03:10.053868+0000","last_undegraded":"2024-09-16T05:03:10.053868+0000","last_fullsized":"2024-09-16T05:03:10.053868+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20024,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692186+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:03:09.692186+0000","last_peered":"2024-09-16T05:03:09.692186+0000","last_clean":"2024-09-16T05:03:09.692186+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:03:09.692186+0000","last_undegraded":"2024-09-16T05:03:09.692186+0000","last_fullsized":"2024-09-16T05:03:09.692186+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20463,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053606+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:03:10.053606+0000","last_peered":"2024-09-16T05:03:10.053606+0000","last_clean":"2024-09-16T05:03:10.053606+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:03:10.053606+0000","last_undegraded":"2024-09-16T05:03:10.053606+0000","last_fullsized":"2024-09-16T05:03:10.053606+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20412,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.694074+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:03:09.694074+0000","last_peered":"2024-09-16T05:03:09.694074+0000","last_clean":"2024-09-16T05:03:09.694074+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:03:09.694074+0000","last_undegraded":"2024-09-16T05:03:09.694074+0000","last_fullsized":"2024-09-16T05:03:09.694074+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20467,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.871337+0000","last_change":"2024-09-16T05:03:09.870297+0000","last_active":"2024-09-16T05:03:09.871337+0000","last_peered":"2024-09-16T05:03:09.871337+0000","last_clean":"2024-09-16T05:03:09.871337+0000","last_became_active":"2024-09-16T05:03:09.695598+0000","last_became_peered":"2024-09-16T05:03:09.695598+0000","last_unstale":"2024-09-16T05:03:09.871337+0000","last_undegraded":"2024-09-16T05:03:09.871337+0000","last_fullsized":"2024-09-16T05:03:09.871337+0000","mapping_epoch":68,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:37.304390+0000","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691796+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:03:09.691796+0000","last_peered":"2024-09-16T05:03:09.691796+0000","last_clean":"2024-09-16T05:03:09.691796+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:03:09.691796+0000","last_undegraded":"2024-09-16T05:03:09.691796+0000","last_fullsized":"2024-09-16T05:03:09.691796+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20281,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693065+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:03:09.693065+0000","last_peered":"2024-09-16T05:03:09.693065+0000","last_clean":"2024-09-16T05:03:09.693065+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:03:09.693065+0000","last_undegraded":"2024-09-16T05:03:09.693065+0000","last_fullsized":"2024-09-16T05:03:09.693065+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691472+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:03:09.691472+0000","last_peered":"2024-09-16T05:03:09.691472+0000","last_clean":"2024-09-16T05:03:09.691472+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:03:09.691472+0000","last_undegraded":"2024-09-16T05:03:09.691472+0000","last_fullsized":"2024-09-16T05:03:09.691472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20344,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692824+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:03:09.692824+0000","last_peered":"2024-09-16T05:03:09.692824+0000","last_clean":"2024-09-16T05:03:09.692824+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:03:09.692824+0000","last_undegraded":"2024-09-16T05:03:09.692824+0000","last_fullsized":"2024-09-16T05:03:09.692824+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053275+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:03:10.053275+0000","last_peered":"2024-09-16T05:03:10.053275+0000","last_clean":"2024-09-16T05:03:10.053275+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:03:10.053275+0000","last_undegraded":"2024-09-16T05:03:10.053275+0000","last_fullsized":"2024-09-16T05:03:10.053275+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27338,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693866+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:03:09.693866+0000","last_peered":"2024-09-16T05:03:09.693866+0000","last_clean":"2024-09-16T05:03:09.693866+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:03:09.693866+0000","last_undegraded":"2024-09-16T05:03:09.693866+0000","last_fullsized":"2024-09-16T05:03:09.693866+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693578+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:03:09.693578+0000","last_peered":"2024-09-16T05:03:09.693578+0000","last_clean":"2024-09-16T05:03:09.693578+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:03:09.693578+0000","last_undegraded":"2024-09-16T05:03:09.693578+0000","last_fullsized":"2024-09-16T05:03:09.693578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26881,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053400+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:03:10.053400+0000","last_peered":"2024-09-16T05:03:10.053400+0000","last_clean":"2024-09-16T05:03:10.053400+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:03:10.053400+0000","last_undegraded":"2024-09-16T05:03:10.053400+0000","last_fullsized":"2024-09-16T05:03:10.053400+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692638+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:03:09.692638+0000","last_peered":"2024-09-16T05:03:09.692638+0000","last_clean":"2024-09-16T05:03:09.692638+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:03:09.692638+0000","last_undegraded":"2024-09-16T05:03:09.692638+0000","last_fullsized":"2024-09-16T05:03:09.692638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26884,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053718+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:03:10.053718+0000","last_peered":"2024-09-16T05:03:10.053718+0000","last_clean":"2024-09-16T05:03:10.053718+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:03:10.053718+0000","last_undegraded":"2024-09-16T05:03:10.053718+0000","last_fullsized":"2024-09-16T05:03:10.053718+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20025,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692648+0000","last_change":"2024-09-16T05:03:09.081130+0000","last_active":"2024-09-16T05:03:09.692648+0000","last_peered":"2024-09-16T05:03:09.692648+0000","last_clean":"2024-09-16T05:03:09.692648+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:03:09.692648+0000","last_undegraded":"2024-09-16T05:03:09.692648+0000","last_fullsized":"2024-09-16T05:03:09.692648+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19876","last_scrub_stamp":"2024-09-16T05:03:09.080970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:03:09.080970+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:19.155082+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693090+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:03:09.693090+0000","last_peered":"2024-09-16T05:03:09.693090+0000","last_clean":"2024-09-16T05:03:09.693090+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:03:09.693090+0000","last_undegraded":"2024-09-16T05:03:09.693090+0000","last_fullsized":"2024-09-16T05:03:09.693090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20687,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692091+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:03:09.692091+0000","last_peered":"2024-09-16T05:03:09.692091+0000","last_clean":"2024-09-16T05:03:09.692091+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:03:09.692091+0000","last_undegraded":"2024-09-16T05:03:09.692091+0000","last_fullsized":"2024-09-16T05:03:09.692091+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:03:10.053647+0000","last_peered":"2024-09-16T05:03:10.053647+0000","last_clean":"2024-09-16T05:03:10.053647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:03:10.053647+0000","last_undegraded":"2024-09-16T05:03:10.053647+0000","last_fullsized":"2024-09-16T05:03:10.053647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27041,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692471+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:03:09.692471+0000","last_peered":"2024-09-16T05:03:09.692471+0000","last_clean":"2024-09-16T05:03:09.692471+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:03:09.692471+0000","last_undegraded":"2024-09-16T05:03:09.692471+0000","last_fullsized":"2024-09-16T05:03:09.692471+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691799+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:03:09.691799+0000","last_peered":"2024-09-16T05:03:09.691799+0000","last_clean":"2024-09-16T05:03:09.691799+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:03:09.691799+0000","last_undegraded":"2024-09-16T05:03:09.691799+0000","last_fullsized":"2024-09-16T05:03:09.691799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692344+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:03:09.692344+0000","last_peered":"2024-09-16T05:03:09.692344+0000","last_clean":"2024-09-16T05:03:09.692344+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:03:09.692344+0000","last_undegraded":"2024-09-16T05:03:09.692344+0000","last_fullsized":"2024-09-16T05:03:09.692344+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20602,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693254+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:03:09.693254+0000","last_peered":"2024-09-16T05:03:09.693254+0000","last_clean":"2024-09-16T05:03:09.693254+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:03:09.693254+0000","last_undegraded":"2024-09-16T05:03:09.693254+0000","last_fullsized":"2024-09-16T05:03:09.693254+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246797,"num_write_kb":424049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34032,"num_bytes_recovered":27324416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":94232576,"data_stored":94225618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425084,"kb_used_data":14376,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946756,"statfs":{"total":96636764160,"available":96201478144,"internally_reserved":0,"allocated":14721024,"data_stored":14349262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593891,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501508,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870332,"statfs":{"total":96636764160,"available":96123219968,"internally_reserved":0,"allocated":573440,"data_stored":194122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487152,"kb_used_data":80668,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884688,"statfs":{"total":96636764160,"available":96137920512,"internally_reserved":0,"allocated":82604032,"data_stored":82224714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":81297408,"data_stored":81293929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12931072,"data_stored":12931072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:12.683 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:12.980 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:12.999 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2024-09-16T05:03:12.621690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2487,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246799,"num_write_kb":424051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34033,"num_bytes_recovered":27324435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413744,"kb_used_data":95604,"kb_used_omap":12,"kb_used_meta":1318067,"kb_avail":281701776,"statfs":{"total":289910292480,"available":288462618624,"internally_reserved":0,"allocated":97898496,"data_stored":96768098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1349700609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-617,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-19875,"num_write_kb":-39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2423,"num_bytes_recovered":-4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001854"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26555,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.695100+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:03:09.695100+0000","last_peered":"2024-09-16T05:03:09.695100+0000","last_clean":"2024-09-16T05:03:09.695100+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:03:09.695100+0000","last_undegraded":"2024-09-16T05:03:09.695100+0000","last_fullsized":"2024-09-16T05:03:09.695100+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20159,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053868+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:03:10.053868+0000","last_peered":"2024-09-16T05:03:10.053868+0000","last_clean":"2024-09-16T05:03:10.053868+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:03:10.053868+0000","last_undegraded":"2024-09-16T05:03:10.053868+0000","last_fullsized":"2024-09-16T05:03:10.053868+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20024,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692186+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:03:09.692186+0000","last_peered":"2024-09-16T05:03:09.692186+0000","last_clean":"2024-09-16T05:03:09.692186+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:03:09.692186+0000","last_undegraded":"2024-09-16T05:03:09.692186+0000","last_fullsized":"2024-09-16T05:03:09.692186+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20463,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053606+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:03:10.053606+0000","last_peered":"2024-09-16T05:03:10.053606+0000","last_clean":"2024-09-16T05:03:10.053606+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:03:10.053606+0000","last_undegraded":"2024-09-16T05:03:10.053606+0000","last_fullsized":"2024-09-16T05:03:10.053606+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20412,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.694074+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:03:09.694074+0000","last_peered":"2024-09-16T05:03:09.694074+0000","last_clean":"2024-09-16T05:03:09.694074+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:03:09.694074+0000","last_undegraded":"2024-09-16T05:03:09.694074+0000","last_fullsized":"2024-09-16T05:03:09.694074+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20467,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.871337+0000","last_change":"2024-09-16T05:03:09.870297+0000","last_active":"2024-09-16T05:03:09.871337+0000","last_peered":"2024-09-16T05:03:09.871337+0000","last_clean":"2024-09-16T05:03:09.871337+0000","last_became_active":"2024-09-16T05:03:09.695598+0000","last_became_peered":"2024-09-16T05:03:09.695598+0000","last_unstale":"2024-09-16T05:03:09.871337+0000","last_undegraded":"2024-09-16T05:03:09.871337+0000","last_fullsized":"2024-09-16T05:03:09.871337+0000","mapping_epoch":68,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:37.304390+0000","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691796+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:03:09.691796+0000","last_peered":"2024-09-16T05:03:09.691796+0000","last_clean":"2024-09-16T05:03:09.691796+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:03:09.691796+0000","last_undegraded":"2024-09-16T05:03:09.691796+0000","last_fullsized":"2024-09-16T05:03:09.691796+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20281,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693065+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:03:09.693065+0000","last_peered":"2024-09-16T05:03:09.693065+0000","last_clean":"2024-09-16T05:03:09.693065+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:03:09.693065+0000","last_undegraded":"2024-09-16T05:03:09.693065+0000","last_fullsized":"2024-09-16T05:03:09.693065+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691472+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:03:09.691472+0000","last_peered":"2024-09-16T05:03:09.691472+0000","last_clean":"2024-09-16T05:03:09.691472+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:03:09.691472+0000","last_undegraded":"2024-09-16T05:03:09.691472+0000","last_fullsized":"2024-09-16T05:03:09.691472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20344,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692824+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:03:09.692824+0000","last_peered":"2024-09-16T05:03:09.692824+0000","last_clean":"2024-09-16T05:03:09.692824+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:03:09.692824+0000","last_undegraded":"2024-09-16T05:03:09.692824+0000","last_fullsized":"2024-09-16T05:03:09.692824+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053275+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:03:10.053275+0000","last_peered":"2024-09-16T05:03:10.053275+0000","last_clean":"2024-09-16T05:03:10.053275+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:03:10.053275+0000","last_undegraded":"2024-09-16T05:03:10.053275+0000","last_fullsized":"2024-09-16T05:03:10.053275+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27338,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693866+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:03:09.693866+0000","last_peered":"2024-09-16T05:03:09.693866+0000","last_clean":"2024-09-16T05:03:09.693866+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:03:09.693866+0000","last_undegraded":"2024-09-16T05:03:09.693866+0000","last_fullsized":"2024-09-16T05:03:09.693866+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693578+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:03:09.693578+0000","last_peered":"2024-09-16T05:03:09.693578+0000","last_clean":"2024-09-16T05:03:09.693578+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:03:09.693578+0000","last_undegraded":"2024-09-16T05:03:09.693578+0000","last_fullsized":"2024-09-16T05:03:09.693578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26881,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053400+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:03:10.053400+0000","last_peered":"2024-09-16T05:03:10.053400+0000","last_clean":"2024-09-16T05:03:10.053400+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:03:10.053400+0000","last_undegraded":"2024-09-16T05:03:10.053400+0000","last_fullsized":"2024-09-16T05:03:10.053400+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692638+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:03:09.692638+0000","last_peered":"2024-09-16T05:03:09.692638+0000","last_clean":"2024-09-16T05:03:09.692638+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:03:09.692638+0000","last_undegraded":"2024-09-16T05:03:09.692638+0000","last_fullsized":"2024-09-16T05:03:09.692638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26884,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053718+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:03:10.053718+0000","last_peered":"2024-09-16T05:03:10.053718+0000","last_clean":"2024-09-16T05:03:10.053718+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:03:10.053718+0000","last_undegraded":"2024-09-16T05:03:10.053718+0000","last_fullsized":"2024-09-16T05:03:10.053718+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20025,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692648+0000","last_change":"2024-09-16T05:03:09.081130+0000","last_active":"2024-09-16T05:03:09.692648+0000","last_peered":"2024-09-16T05:03:09.692648+0000","last_clean":"2024-09-16T05:03:09.692648+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:03:09.692648+0000","last_undegraded":"2024-09-16T05:03:09.692648+0000","last_fullsized":"2024-09-16T05:03:09.692648+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19876","last_scrub_stamp":"2024-09-16T05:03:09.080970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:03:09.080970+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:19.155082+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693090+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:03:09.693090+0000","last_peered":"2024-09-16T05:03:09.693090+0000","last_clean":"2024-09-16T05:03:09.693090+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:03:09.693090+0000","last_undegraded":"2024-09-16T05:03:09.693090+0000","last_fullsized":"2024-09-16T05:03:09.693090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20687,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692091+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:03:09.692091+0000","last_peered":"2024-09-16T05:03:09.692091+0000","last_clean":"2024-09-16T05:03:09.692091+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:03:09.692091+0000","last_undegraded":"2024-09-16T05:03:09.692091+0000","last_fullsized":"2024-09-16T05:03:09.692091+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:03:10.053647+0000","last_peered":"2024-09-16T05:03:10.053647+0000","last_clean":"2024-09-16T05:03:10.053647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:03:10.053647+0000","last_undegraded":"2024-09-16T05:03:10.053647+0000","last_fullsized":"2024-09-16T05:03:10.053647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27041,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692471+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:03:09.692471+0000","last_peered":"2024-09-16T05:03:09.692471+0000","last_clean":"2024-09-16T05:03:09.692471+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:03:09.692471+0000","last_undegraded":"2024-09-16T05:03:09.692471+0000","last_fullsized":"2024-09-16T05:03:09.692471+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691799+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:03:09.691799+0000","last_peered":"2024-09-16T05:03:09.691799+0000","last_clean":"2024-09-16T05:03:09.691799+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:03:09.691799+0000","last_undegraded":"2024-09-16T05:03:09.691799+0000","last_fullsized":"2024-09-16T05:03:09.691799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692344+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:03:09.692344+0000","last_peered":"2024-09-16T05:03:09.692344+0000","last_clean":"2024-09-16T05:03:09.692344+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:03:09.692344+0000","last_undegraded":"2024-09-16T05:03:09.692344+0000","last_fullsized":"2024-09-16T05:03:09.692344+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20602,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693254+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:03:09.693254+0000","last_peered":"2024-09-16T05:03:09.693254+0000","last_clean":"2024-09-16T05:03:09.693254+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:03:09.693254+0000","last_undegraded":"2024-09-16T05:03:09.693254+0000","last_fullsized":"2024-09-16T05:03:09.693254+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246797,"num_write_kb":424049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34032,"num_bytes_recovered":27324416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":94232576,"data_stored":94225618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425084,"kb_used_data":14376,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946756,"statfs":{"total":96636764160,"available":96201478144,"internally_reserved":0,"allocated":14721024,"data_stored":14349262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593891,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501508,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870332,"statfs":{"total":96636764160,"available":96123219968,"internally_reserved":0,"allocated":573440,"data_stored":194122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487152,"kb_used_data":80668,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884688,"statfs":{"total":96636764160,"available":96137920512,"internally_reserved":0,"allocated":82604032,"data_stored":82224714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":81297408,"data_stored":81293929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12931072,"data_stored":12931072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:13.000 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-16T05:03:13.000 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T05:03:13.009 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T05:03:13.009 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:13.297 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:13.313 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2024-09-16T05:03:12.621690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2487,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246799,"num_write_kb":424051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34033,"num_bytes_recovered":27324435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413744,"kb_used_data":95604,"kb_used_omap":12,"kb_used_meta":1318067,"kb_avail":281701776,"statfs":{"total":289910292480,"available":288462618624,"internally_reserved":0,"allocated":97898496,"data_stored":96768098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1349700609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-617,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-19875,"num_write_kb":-39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2423,"num_bytes_recovered":-4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001854"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26555,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.695100+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:03:09.695100+0000","last_peered":"2024-09-16T05:03:09.695100+0000","last_clean":"2024-09-16T05:03:09.695100+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:03:09.695100+0000","last_undegraded":"2024-09-16T05:03:09.695100+0000","last_fullsized":"2024-09-16T05:03:09.695100+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20159,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053868+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:03:10.053868+0000","last_peered":"2024-09-16T05:03:10.053868+0000","last_clean":"2024-09-16T05:03:10.053868+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:03:10.053868+0000","last_undegraded":"2024-09-16T05:03:10.053868+0000","last_fullsized":"2024-09-16T05:03:10.053868+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20024,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692186+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:03:09.692186+0000","last_peered":"2024-09-16T05:03:09.692186+0000","last_clean":"2024-09-16T05:03:09.692186+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:03:09.692186+0000","last_undegraded":"2024-09-16T05:03:09.692186+0000","last_fullsized":"2024-09-16T05:03:09.692186+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20463,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053606+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:03:10.053606+0000","last_peered":"2024-09-16T05:03:10.053606+0000","last_clean":"2024-09-16T05:03:10.053606+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:03:10.053606+0000","last_undegraded":"2024-09-16T05:03:10.053606+0000","last_fullsized":"2024-09-16T05:03:10.053606+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20412,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.694074+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:03:09.694074+0000","last_peered":"2024-09-16T05:03:09.694074+0000","last_clean":"2024-09-16T05:03:09.694074+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:03:09.694074+0000","last_undegraded":"2024-09-16T05:03:09.694074+0000","last_fullsized":"2024-09-16T05:03:09.694074+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20467,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.871337+0000","last_change":"2024-09-16T05:03:09.870297+0000","last_active":"2024-09-16T05:03:09.871337+0000","last_peered":"2024-09-16T05:03:09.871337+0000","last_clean":"2024-09-16T05:03:09.871337+0000","last_became_active":"2024-09-16T05:03:09.695598+0000","last_became_peered":"2024-09-16T05:03:09.695598+0000","last_unstale":"2024-09-16T05:03:09.871337+0000","last_undegraded":"2024-09-16T05:03:09.871337+0000","last_fullsized":"2024-09-16T05:03:09.871337+0000","mapping_epoch":68,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:37.304390+0000","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691796+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:03:09.691796+0000","last_peered":"2024-09-16T05:03:09.691796+0000","last_clean":"2024-09-16T05:03:09.691796+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:03:09.691796+0000","last_undegraded":"2024-09-16T05:03:09.691796+0000","last_fullsized":"2024-09-16T05:03:09.691796+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20281,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693065+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:03:09.693065+0000","last_peered":"2024-09-16T05:03:09.693065+0000","last_clean":"2024-09-16T05:03:09.693065+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:03:09.693065+0000","last_undegraded":"2024-09-16T05:03:09.693065+0000","last_fullsized":"2024-09-16T05:03:09.693065+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691472+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:03:09.691472+0000","last_peered":"2024-09-16T05:03:09.691472+0000","last_clean":"2024-09-16T05:03:09.691472+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:03:09.691472+0000","last_undegraded":"2024-09-16T05:03:09.691472+0000","last_fullsized":"2024-09-16T05:03:09.691472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20344,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692824+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:03:09.692824+0000","last_peered":"2024-09-16T05:03:09.692824+0000","last_clean":"2024-09-16T05:03:09.692824+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:03:09.692824+0000","last_undegraded":"2024-09-16T05:03:09.692824+0000","last_fullsized":"2024-09-16T05:03:09.692824+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053275+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:03:10.053275+0000","last_peered":"2024-09-16T05:03:10.053275+0000","last_clean":"2024-09-16T05:03:10.053275+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:03:10.053275+0000","last_undegraded":"2024-09-16T05:03:10.053275+0000","last_fullsized":"2024-09-16T05:03:10.053275+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27338,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693866+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:03:09.693866+0000","last_peered":"2024-09-16T05:03:09.693866+0000","last_clean":"2024-09-16T05:03:09.693866+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:03:09.693866+0000","last_undegraded":"2024-09-16T05:03:09.693866+0000","last_fullsized":"2024-09-16T05:03:09.693866+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693578+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:03:09.693578+0000","last_peered":"2024-09-16T05:03:09.693578+0000","last_clean":"2024-09-16T05:03:09.693578+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:03:09.693578+0000","last_undegraded":"2024-09-16T05:03:09.693578+0000","last_fullsized":"2024-09-16T05:03:09.693578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26881,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053400+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:03:10.053400+0000","last_peered":"2024-09-16T05:03:10.053400+0000","last_clean":"2024-09-16T05:03:10.053400+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:03:10.053400+0000","last_undegraded":"2024-09-16T05:03:10.053400+0000","last_fullsized":"2024-09-16T05:03:10.053400+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692638+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:03:09.692638+0000","last_peered":"2024-09-16T05:03:09.692638+0000","last_clean":"2024-09-16T05:03:09.692638+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:03:09.692638+0000","last_undegraded":"2024-09-16T05:03:09.692638+0000","last_fullsized":"2024-09-16T05:03:09.692638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26884,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053718+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:03:10.053718+0000","last_peered":"2024-09-16T05:03:10.053718+0000","last_clean":"2024-09-16T05:03:10.053718+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:03:10.053718+0000","last_undegraded":"2024-09-16T05:03:10.053718+0000","last_fullsized":"2024-09-16T05:03:10.053718+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20025,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692648+0000","last_change":"2024-09-16T05:03:09.081130+0000","last_active":"2024-09-16T05:03:09.692648+0000","last_peered":"2024-09-16T05:03:09.692648+0000","last_clean":"2024-09-16T05:03:09.692648+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:03:09.692648+0000","last_undegraded":"2024-09-16T05:03:09.692648+0000","last_fullsized":"2024-09-16T05:03:09.692648+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19876","last_scrub_stamp":"2024-09-16T05:03:09.080970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:03:09.080970+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:19.155082+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693090+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:03:09.693090+0000","last_peered":"2024-09-16T05:03:09.693090+0000","last_clean":"2024-09-16T05:03:09.693090+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:03:09.693090+0000","last_undegraded":"2024-09-16T05:03:09.693090+0000","last_fullsized":"2024-09-16T05:03:09.693090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20687,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692091+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:03:09.692091+0000","last_peered":"2024-09-16T05:03:09.692091+0000","last_clean":"2024-09-16T05:03:09.692091+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:03:09.692091+0000","last_undegraded":"2024-09-16T05:03:09.692091+0000","last_fullsized":"2024-09-16T05:03:09.692091+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:03:10.053647+0000","last_peered":"2024-09-16T05:03:10.053647+0000","last_clean":"2024-09-16T05:03:10.053647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:03:10.053647+0000","last_undegraded":"2024-09-16T05:03:10.053647+0000","last_fullsized":"2024-09-16T05:03:10.053647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27041,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692471+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:03:09.692471+0000","last_peered":"2024-09-16T05:03:09.692471+0000","last_clean":"2024-09-16T05:03:09.692471+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:03:09.692471+0000","last_undegraded":"2024-09-16T05:03:09.692471+0000","last_fullsized":"2024-09-16T05:03:09.692471+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691799+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:03:09.691799+0000","last_peered":"2024-09-16T05:03:09.691799+0000","last_clean":"2024-09-16T05:03:09.691799+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:03:09.691799+0000","last_undegraded":"2024-09-16T05:03:09.691799+0000","last_fullsized":"2024-09-16T05:03:09.691799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692344+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:03:09.692344+0000","last_peered":"2024-09-16T05:03:09.692344+0000","last_clean":"2024-09-16T05:03:09.692344+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:03:09.692344+0000","last_undegraded":"2024-09-16T05:03:09.692344+0000","last_fullsized":"2024-09-16T05:03:09.692344+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20602,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693254+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:03:09.693254+0000","last_peered":"2024-09-16T05:03:09.693254+0000","last_clean":"2024-09-16T05:03:09.693254+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:03:09.693254+0000","last_undegraded":"2024-09-16T05:03:09.693254+0000","last_fullsized":"2024-09-16T05:03:09.693254+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246797,"num_write_kb":424049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34032,"num_bytes_recovered":27324416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":94232576,"data_stored":94225618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425084,"kb_used_data":14376,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946756,"statfs":{"total":96636764160,"available":96201478144,"internally_reserved":0,"allocated":14721024,"data_stored":14349262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593891,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501508,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870332,"statfs":{"total":96636764160,"available":96123219968,"internally_reserved":0,"allocated":573440,"data_stored":194122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487152,"kb_used_data":80668,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884688,"statfs":{"total":96636764160,"available":96137920512,"internally_reserved":0,"allocated":82604032,"data_stored":82224714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":81297408,"data_stored":81293929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12931072,"data_stored":12931072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:13.314 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:13.608 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:13.629 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2024-09-16T05:03:12.621690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2487,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246799,"num_write_kb":424051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34033,"num_bytes_recovered":27324435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413744,"kb_used_data":95604,"kb_used_omap":12,"kb_used_meta":1318067,"kb_avail":281701776,"statfs":{"total":289910292480,"available":288462618624,"internally_reserved":0,"allocated":97898496,"data_stored":96768098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1349700609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-617,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-19875,"num_write_kb":-39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2423,"num_bytes_recovered":-4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001854"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26555,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.695100+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:03:09.695100+0000","last_peered":"2024-09-16T05:03:09.695100+0000","last_clean":"2024-09-16T05:03:09.695100+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:03:09.695100+0000","last_undegraded":"2024-09-16T05:03:09.695100+0000","last_fullsized":"2024-09-16T05:03:09.695100+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20159,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053868+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:03:10.053868+0000","last_peered":"2024-09-16T05:03:10.053868+0000","last_clean":"2024-09-16T05:03:10.053868+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:03:10.053868+0000","last_undegraded":"2024-09-16T05:03:10.053868+0000","last_fullsized":"2024-09-16T05:03:10.053868+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20024,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692186+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:03:09.692186+0000","last_peered":"2024-09-16T05:03:09.692186+0000","last_clean":"2024-09-16T05:03:09.692186+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:03:09.692186+0000","last_undegraded":"2024-09-16T05:03:09.692186+0000","last_fullsized":"2024-09-16T05:03:09.692186+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20463,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053606+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:03:10.053606+0000","last_peered":"2024-09-16T05:03:10.053606+0000","last_clean":"2024-09-16T05:03:10.053606+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:03:10.053606+0000","last_undegraded":"2024-09-16T05:03:10.053606+0000","last_fullsized":"2024-09-16T05:03:10.053606+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20412,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.694074+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:03:09.694074+0000","last_peered":"2024-09-16T05:03:09.694074+0000","last_clean":"2024-09-16T05:03:09.694074+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:03:09.694074+0000","last_undegraded":"2024-09-16T05:03:09.694074+0000","last_fullsized":"2024-09-16T05:03:09.694074+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20467,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.871337+0000","last_change":"2024-09-16T05:03:09.870297+0000","last_active":"2024-09-16T05:03:09.871337+0000","last_peered":"2024-09-16T05:03:09.871337+0000","last_clean":"2024-09-16T05:03:09.871337+0000","last_became_active":"2024-09-16T05:03:09.695598+0000","last_became_peered":"2024-09-16T05:03:09.695598+0000","last_unstale":"2024-09-16T05:03:09.871337+0000","last_undegraded":"2024-09-16T05:03:09.871337+0000","last_fullsized":"2024-09-16T05:03:09.871337+0000","mapping_epoch":68,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:37.304390+0000","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691796+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:03:09.691796+0000","last_peered":"2024-09-16T05:03:09.691796+0000","last_clean":"2024-09-16T05:03:09.691796+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:03:09.691796+0000","last_undegraded":"2024-09-16T05:03:09.691796+0000","last_fullsized":"2024-09-16T05:03:09.691796+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20281,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693065+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:03:09.693065+0000","last_peered":"2024-09-16T05:03:09.693065+0000","last_clean":"2024-09-16T05:03:09.693065+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:03:09.693065+0000","last_undegraded":"2024-09-16T05:03:09.693065+0000","last_fullsized":"2024-09-16T05:03:09.693065+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691472+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:03:09.691472+0000","last_peered":"2024-09-16T05:03:09.691472+0000","last_clean":"2024-09-16T05:03:09.691472+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:03:09.691472+0000","last_undegraded":"2024-09-16T05:03:09.691472+0000","last_fullsized":"2024-09-16T05:03:09.691472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20344,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692824+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:03:09.692824+0000","last_peered":"2024-09-16T05:03:09.692824+0000","last_clean":"2024-09-16T05:03:09.692824+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:03:09.692824+0000","last_undegraded":"2024-09-16T05:03:09.692824+0000","last_fullsized":"2024-09-16T05:03:09.692824+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053275+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:03:10.053275+0000","last_peered":"2024-09-16T05:03:10.053275+0000","last_clean":"2024-09-16T05:03:10.053275+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:03:10.053275+0000","last_undegraded":"2024-09-16T05:03:10.053275+0000","last_fullsized":"2024-09-16T05:03:10.053275+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27338,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693866+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:03:09.693866+0000","last_peered":"2024-09-16T05:03:09.693866+0000","last_clean":"2024-09-16T05:03:09.693866+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:03:09.693866+0000","last_undegraded":"2024-09-16T05:03:09.693866+0000","last_fullsized":"2024-09-16T05:03:09.693866+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693578+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:03:09.693578+0000","last_peered":"2024-09-16T05:03:09.693578+0000","last_clean":"2024-09-16T05:03:09.693578+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:03:09.693578+0000","last_undegraded":"2024-09-16T05:03:09.693578+0000","last_fullsized":"2024-09-16T05:03:09.693578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26881,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053400+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:03:10.053400+0000","last_peered":"2024-09-16T05:03:10.053400+0000","last_clean":"2024-09-16T05:03:10.053400+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:03:10.053400+0000","last_undegraded":"2024-09-16T05:03:10.053400+0000","last_fullsized":"2024-09-16T05:03:10.053400+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692638+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:03:09.692638+0000","last_peered":"2024-09-16T05:03:09.692638+0000","last_clean":"2024-09-16T05:03:09.692638+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:03:09.692638+0000","last_undegraded":"2024-09-16T05:03:09.692638+0000","last_fullsized":"2024-09-16T05:03:09.692638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26884,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053718+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:03:10.053718+0000","last_peered":"2024-09-16T05:03:10.053718+0000","last_clean":"2024-09-16T05:03:10.053718+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:03:10.053718+0000","last_undegraded":"2024-09-16T05:03:10.053718+0000","last_fullsized":"2024-09-16T05:03:10.053718+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20025,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692648+0000","last_change":"2024-09-16T05:03:09.081130+0000","last_active":"2024-09-16T05:03:09.692648+0000","last_peered":"2024-09-16T05:03:09.692648+0000","last_clean":"2024-09-16T05:03:09.692648+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:03:09.692648+0000","last_undegraded":"2024-09-16T05:03:09.692648+0000","last_fullsized":"2024-09-16T05:03:09.692648+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19876","last_scrub_stamp":"2024-09-16T05:03:09.080970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:03:09.080970+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:19.155082+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693090+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:03:09.693090+0000","last_peered":"2024-09-16T05:03:09.693090+0000","last_clean":"2024-09-16T05:03:09.693090+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:03:09.693090+0000","last_undegraded":"2024-09-16T05:03:09.693090+0000","last_fullsized":"2024-09-16T05:03:09.693090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20687,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692091+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:03:09.692091+0000","last_peered":"2024-09-16T05:03:09.692091+0000","last_clean":"2024-09-16T05:03:09.692091+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:03:09.692091+0000","last_undegraded":"2024-09-16T05:03:09.692091+0000","last_fullsized":"2024-09-16T05:03:09.692091+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:03:10.053647+0000","last_peered":"2024-09-16T05:03:10.053647+0000","last_clean":"2024-09-16T05:03:10.053647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:03:10.053647+0000","last_undegraded":"2024-09-16T05:03:10.053647+0000","last_fullsized":"2024-09-16T05:03:10.053647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27041,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692471+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:03:09.692471+0000","last_peered":"2024-09-16T05:03:09.692471+0000","last_clean":"2024-09-16T05:03:09.692471+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:03:09.692471+0000","last_undegraded":"2024-09-16T05:03:09.692471+0000","last_fullsized":"2024-09-16T05:03:09.692471+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691799+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:03:09.691799+0000","last_peered":"2024-09-16T05:03:09.691799+0000","last_clean":"2024-09-16T05:03:09.691799+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:03:09.691799+0000","last_undegraded":"2024-09-16T05:03:09.691799+0000","last_fullsized":"2024-09-16T05:03:09.691799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692344+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:03:09.692344+0000","last_peered":"2024-09-16T05:03:09.692344+0000","last_clean":"2024-09-16T05:03:09.692344+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:03:09.692344+0000","last_undegraded":"2024-09-16T05:03:09.692344+0000","last_fullsized":"2024-09-16T05:03:09.692344+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20602,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693254+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:03:09.693254+0000","last_peered":"2024-09-16T05:03:09.693254+0000","last_clean":"2024-09-16T05:03:09.693254+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:03:09.693254+0000","last_undegraded":"2024-09-16T05:03:09.693254+0000","last_fullsized":"2024-09-16T05:03:09.693254+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246797,"num_write_kb":424049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34032,"num_bytes_recovered":27324416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":94232576,"data_stored":94225618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425084,"kb_used_data":14376,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946756,"statfs":{"total":96636764160,"available":96201478144,"internally_reserved":0,"allocated":14721024,"data_stored":14349262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593891,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501508,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870332,"statfs":{"total":96636764160,"available":96123219968,"internally_reserved":0,"allocated":573440,"data_stored":194122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487152,"kb_used_data":80668,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884688,"statfs":{"total":96636764160,"available":96137920512,"internally_reserved":0,"allocated":82604032,"data_stored":82224714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":81297408,"data_stored":81293929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12931072,"data_stored":12931072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:13.631 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T05:03:13.631 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:13.926 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:13.927 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:13.943 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2024-09-16T05:03:12.621690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2487,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246799,"num_write_kb":424051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34033,"num_bytes_recovered":27324435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413744,"kb_used_data":95604,"kb_used_omap":12,"kb_used_meta":1318067,"kb_avail":281701776,"statfs":{"total":289910292480,"available":288462618624,"internally_reserved":0,"allocated":97898496,"data_stored":96768098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1349700609},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-617,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-987,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-19875,"num_write_kb":-39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2423,"num_bytes_recovered":-4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001854"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26555,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.695100+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:03:09.695100+0000","last_peered":"2024-09-16T05:03:09.695100+0000","last_clean":"2024-09-16T05:03:09.695100+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:03:09.695100+0000","last_undegraded":"2024-09-16T05:03:09.695100+0000","last_fullsized":"2024-09-16T05:03:09.695100+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20159,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053868+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:03:10.053868+0000","last_peered":"2024-09-16T05:03:10.053868+0000","last_clean":"2024-09-16T05:03:10.053868+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:03:10.053868+0000","last_undegraded":"2024-09-16T05:03:10.053868+0000","last_fullsized":"2024-09-16T05:03:10.053868+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20024,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692186+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:03:09.692186+0000","last_peered":"2024-09-16T05:03:09.692186+0000","last_clean":"2024-09-16T05:03:09.692186+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:03:09.692186+0000","last_undegraded":"2024-09-16T05:03:09.692186+0000","last_fullsized":"2024-09-16T05:03:09.692186+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20463,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053606+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:03:10.053606+0000","last_peered":"2024-09-16T05:03:10.053606+0000","last_clean":"2024-09-16T05:03:10.053606+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:03:10.053606+0000","last_undegraded":"2024-09-16T05:03:10.053606+0000","last_fullsized":"2024-09-16T05:03:10.053606+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20412,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.694074+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:03:09.694074+0000","last_peered":"2024-09-16T05:03:09.694074+0000","last_clean":"2024-09-16T05:03:09.694074+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:03:09.694074+0000","last_undegraded":"2024-09-16T05:03:09.694074+0000","last_fullsized":"2024-09-16T05:03:09.694074+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20467,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.871337+0000","last_change":"2024-09-16T05:03:09.870297+0000","last_active":"2024-09-16T05:03:09.871337+0000","last_peered":"2024-09-16T05:03:09.871337+0000","last_clean":"2024-09-16T05:03:09.871337+0000","last_became_active":"2024-09-16T05:03:09.695598+0000","last_became_peered":"2024-09-16T05:03:09.695598+0000","last_unstale":"2024-09-16T05:03:09.871337+0000","last_undegraded":"2024-09-16T05:03:09.871337+0000","last_fullsized":"2024-09-16T05:03:09.871337+0000","mapping_epoch":68,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:37.304390+0000","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691796+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:03:09.691796+0000","last_peered":"2024-09-16T05:03:09.691796+0000","last_clean":"2024-09-16T05:03:09.691796+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:03:09.691796+0000","last_undegraded":"2024-09-16T05:03:09.691796+0000","last_fullsized":"2024-09-16T05:03:09.691796+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20281,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693065+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:03:09.693065+0000","last_peered":"2024-09-16T05:03:09.693065+0000","last_clean":"2024-09-16T05:03:09.693065+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:03:09.693065+0000","last_undegraded":"2024-09-16T05:03:09.693065+0000","last_fullsized":"2024-09-16T05:03:09.693065+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691472+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:03:09.691472+0000","last_peered":"2024-09-16T05:03:09.691472+0000","last_clean":"2024-09-16T05:03:09.691472+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:03:09.691472+0000","last_undegraded":"2024-09-16T05:03:09.691472+0000","last_fullsized":"2024-09-16T05:03:09.691472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20344,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692824+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:03:09.692824+0000","last_peered":"2024-09-16T05:03:09.692824+0000","last_clean":"2024-09-16T05:03:09.692824+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:03:09.692824+0000","last_undegraded":"2024-09-16T05:03:09.692824+0000","last_fullsized":"2024-09-16T05:03:09.692824+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053275+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:03:10.053275+0000","last_peered":"2024-09-16T05:03:10.053275+0000","last_clean":"2024-09-16T05:03:10.053275+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:03:10.053275+0000","last_undegraded":"2024-09-16T05:03:10.053275+0000","last_fullsized":"2024-09-16T05:03:10.053275+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27338,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693866+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:03:09.693866+0000","last_peered":"2024-09-16T05:03:09.693866+0000","last_clean":"2024-09-16T05:03:09.693866+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:03:09.693866+0000","last_undegraded":"2024-09-16T05:03:09.693866+0000","last_fullsized":"2024-09-16T05:03:09.693866+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693578+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:03:09.693578+0000","last_peered":"2024-09-16T05:03:09.693578+0000","last_clean":"2024-09-16T05:03:09.693578+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:03:09.693578+0000","last_undegraded":"2024-09-16T05:03:09.693578+0000","last_fullsized":"2024-09-16T05:03:09.693578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26881,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053400+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:03:10.053400+0000","last_peered":"2024-09-16T05:03:10.053400+0000","last_clean":"2024-09-16T05:03:10.053400+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:03:10.053400+0000","last_undegraded":"2024-09-16T05:03:10.053400+0000","last_fullsized":"2024-09-16T05:03:10.053400+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692638+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:03:09.692638+0000","last_peered":"2024-09-16T05:03:09.692638+0000","last_clean":"2024-09-16T05:03:09.692638+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:03:09.692638+0000","last_undegraded":"2024-09-16T05:03:09.692638+0000","last_fullsized":"2024-09-16T05:03:09.692638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26884,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053718+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:03:10.053718+0000","last_peered":"2024-09-16T05:03:10.053718+0000","last_clean":"2024-09-16T05:03:10.053718+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:03:10.053718+0000","last_undegraded":"2024-09-16T05:03:10.053718+0000","last_fullsized":"2024-09-16T05:03:10.053718+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20025,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692648+0000","last_change":"2024-09-16T05:03:09.081130+0000","last_active":"2024-09-16T05:03:09.692648+0000","last_peered":"2024-09-16T05:03:09.692648+0000","last_clean":"2024-09-16T05:03:09.692648+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:03:09.692648+0000","last_undegraded":"2024-09-16T05:03:09.692648+0000","last_fullsized":"2024-09-16T05:03:09.692648+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19876","last_scrub_stamp":"2024-09-16T05:03:09.080970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:03:09.080970+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:19.155082+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693090+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:03:09.693090+0000","last_peered":"2024-09-16T05:03:09.693090+0000","last_clean":"2024-09-16T05:03:09.693090+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:03:09.693090+0000","last_undegraded":"2024-09-16T05:03:09.693090+0000","last_fullsized":"2024-09-16T05:03:09.693090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20687,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692091+0000","last_change":"2024-09-16T04:59:25.904348+0000","last_active":"2024-09-16T05:03:09.692091+0000","last_peered":"2024-09-16T05:03:09.692091+0000","last_clean":"2024-09-16T05:03:09.692091+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:03:09.692091+0000","last_undegraded":"2024-09-16T05:03:09.692091+0000","last_fullsized":"2024-09-16T05:03:09.692091+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":399,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20529,"num_write_kb":41052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3521,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:03:10.053647+0000","last_peered":"2024-09-16T05:03:10.053647+0000","last_clean":"2024-09-16T05:03:10.053647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:03:10.053647+0000","last_undegraded":"2024-09-16T05:03:10.053647+0000","last_fullsized":"2024-09-16T05:03:10.053647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27041,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692471+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:03:09.692471+0000","last_peered":"2024-09-16T05:03:09.692471+0000","last_clean":"2024-09-16T05:03:09.692471+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:03:09.692471+0000","last_undegraded":"2024-09-16T05:03:09.692471+0000","last_fullsized":"2024-09-16T05:03:09.692471+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691799+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:03:09.691799+0000","last_peered":"2024-09-16T05:03:09.691799+0000","last_clean":"2024-09-16T05:03:09.691799+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:03:09.691799+0000","last_undegraded":"2024-09-16T05:03:09.691799+0000","last_fullsized":"2024-09-16T05:03:09.691799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692344+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:03:09.692344+0000","last_peered":"2024-09-16T05:03:09.692344+0000","last_clean":"2024-09-16T05:03:09.692344+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:03:09.692344+0000","last_undegraded":"2024-09-16T05:03:09.692344+0000","last_fullsized":"2024-09-16T05:03:09.692344+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20602,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693254+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:03:09.693254+0000","last_peered":"2024-09-16T05:03:09.693254+0000","last_clean":"2024-09-16T05:03:09.693254+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:03:09.693254+0000","last_undegraded":"2024-09-16T05:03:09.693254+0000","last_fullsized":"2024-09-16T05:03:09.693254+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246797,"num_write_kb":424049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34032,"num_bytes_recovered":27324416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":94232576,"data_stored":94225618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425084,"kb_used_data":14376,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93946756,"statfs":{"total":96636764160,"available":96201478144,"internally_reserved":0,"allocated":14721024,"data_stored":14349262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":420540073},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593891,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501508,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870332,"statfs":{"total":96636764160,"available":96123219968,"internally_reserved":0,"allocated":573440,"data_stored":194122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487152,"kb_used_data":80668,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93884688,"statfs":{"total":96636764160,"available":96137920512,"internally_reserved":0,"allocated":82604032,"data_stored":82224714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":81297408,"data_stored":81293929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12931072,"data_stored":12931072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:13.945 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T05:03:14.237 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:14.237 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":69,"fsid":"b9a92eba-c518-4afa-ae65-ed2825dd0c06","created":"2024-09-16T04:47:27.160076+0000","modified":"2024-09-16T05:03:09.683560+0000","last_up_change":"2024-09-16T04:49:35.617843+0000","last_in_change":"2024-09-16T04:49:28.546152+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-16T04:47:42.612259+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-16T04:47:55.339654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7fa4b8a1-d218-47cb-adc4-469674c829fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":21,"up_from":25,"up_thru":67,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6800","nonce":2560862748}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6801","nonce":2560862748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6803","nonce":2560862748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6802","nonce":2560862748}]},"public_addr":"172.21.15.12:6800/2560862748","cluster_addr":"172.21.15.12:6801/2560862748","heartbeat_back_addr":"172.21.15.12:6803/2560862748","heartbeat_front_addr":"172.21.15.12:6802/2560862748","state":["exists","up"]},{"osd":1,"uuid":"7234f3ae-2cf3-4dd4-8c21-4f170976dbf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":43,"up_thru":68,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6808","nonce":183716785}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6809","nonce":183716785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6811","nonce":183716785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6810","nonce":183716785}]},"public_addr":"172.21.15.12:6808/183716785","cluster_addr":"172.21.15.12:6809/183716785","heartbeat_back_addr":"172.21.15.12:6811/183716785","heartbeat_front_addr":"172.21.15.12:6810/183716785","state":["exists","up"]},{"osd":2,"uuid":"919a351e-ecee-4e06-bff1-ae68c80d320f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":18,"up_from":31,"up_thru":65,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6804","nonce":1526510468}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6805","nonce":1526510468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6807","nonce":1526510468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.12:6806","nonce":1526510468}]},"public_addr":"172.21.15.12:6804/1526510468","cluster_addr":"172.21.15.12:6805/1526510468","heartbeat_back_addr":"172.21.15.12:6807/1526510468","heartbeat_front_addr":"172.21.15.12:6806/1526510468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.109010+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:35.949029+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T04:47:36.023975+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"1.0","mappings":[{"from":0,"to":2}]},{"pgid":"2.6","mappings":[{"from":0,"to":2}]},{"pgid":"2.7","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T05:03:15.255 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T05:03:15.255 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T05:03:15.416 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T05:03:15.416 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T05:03:15.416 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T05:03:15.431 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T05:03:15.720 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T05:03:15.738 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T05:03:15.738 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T05:03:15.879 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T05:03:15.879 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-16T05:03:15.879 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T05:03:15.894 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T05:03:16.176 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T05:03:16.194 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T05:03:16.194 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T05:03:16.329 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-16T05:03:16.330 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T05:03:16.330 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-16T05:03:16.344 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T05:03:16.629 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T05:03:16.647 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:16.937 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:16.938 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:16.953 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2024-09-16T05:03:16.622606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226270,"num_write_kb":382999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30512,"num_bytes_recovered":27320339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1400504,"kb_used_data":78204,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281715016,"statfs":{"total":289910292480,"available":288476176384,"internally_reserved":0,"allocated":80080896,"data_stored":78934310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1353960449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2468,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-434,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-40404,"num_write_kb":-80801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-6497,"num_bytes_recovered":-8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001829"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26555,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.695100+0000","last_change":"2024-09-16T04:58:54.924942+0000","last_active":"2024-09-16T05:03:09.695100+0000","last_peered":"2024-09-16T05:03:09.695100+0000","last_clean":"2024-09-16T05:03:09.695100+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:03:09.695100+0000","last_undegraded":"2024-09-16T05:03:09.695100+0000","last_fullsized":"2024-09-16T05:03:09.695100+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:00.959092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20159,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053868+0000","last_change":"2024-09-16T04:49:26.331586+0000","last_active":"2024-09-16T05:03:10.053868+0000","last_peered":"2024-09-16T05:03:10.053868+0000","last_clean":"2024-09-16T05:03:10.053868+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:03:10.053868+0000","last_undegraded":"2024-09-16T05:03:10.053868+0000","last_fullsized":"2024-09-16T05:03:10.053868+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3054","last_scrub_stamp":"2024-09-16T04:49:26.331430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:26.331430+0000","objects_scrubbed":3025,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:27:07.242424+0000","scrub_duration":1079,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20024,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692186+0000","last_change":"2024-09-16T05:02:39.929857+0000","last_active":"2024-09-16T05:03:09.692186+0000","last_peered":"2024-09-16T05:03:09.692186+0000","last_clean":"2024-09-16T05:03:09.692186+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:03:09.692186+0000","last_undegraded":"2024-09-16T05:03:09.692186+0000","last_fullsized":"2024-09-16T05:03:09.692186+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3261","last_scrub_stamp":"2024-09-16T04:49:33.068296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.068296+0000","objects_scrubbed":3257,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.399711+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20463,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053606+0000","last_change":"2024-09-16T04:48:46.454392+0000","last_active":"2024-09-16T05:03:10.053606+0000","last_peered":"2024-09-16T05:03:10.053606+0000","last_clean":"2024-09-16T05:03:10.053606+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:03:10.053606+0000","last_undegraded":"2024-09-16T05:03:10.053606+0000","last_fullsized":"2024-09-16T05:03:10.053606+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:47:55.367805+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:56.397515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20412,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.694074+0000","last_change":"2024-09-16T04:49:28.644881+0000","last_active":"2024-09-16T05:03:09.694074+0000","last_peered":"2024-09-16T05:03:09.694074+0000","last_clean":"2024-09-16T05:03:09.694074+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:03:09.694074+0000","last_undegraded":"2024-09-16T05:03:09.694074+0000","last_fullsized":"2024-09-16T05:03:09.694074+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'3168","last_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_deep_scrub":"37'3168","last_deep_scrub_stamp":"2024-09-16T04:49:28.644685+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.644685+0000","objects_scrubbed":2938,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:11.869452+0000","scrub_duration":6495,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20467,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.871337+0000","last_change":"2024-09-16T05:03:09.870297+0000","last_active":"2024-09-16T05:03:09.871337+0000","last_peered":"2024-09-16T05:03:09.871337+0000","last_clean":"2024-09-16T05:03:09.871337+0000","last_became_active":"2024-09-16T05:03:09.695598+0000","last_became_peered":"2024-09-16T05:03:09.695598+0000","last_unstale":"2024-09-16T05:03:09.871337+0000","last_undegraded":"2024-09-16T05:03:09.871337+0000","last_fullsized":"2024-09-16T05:03:09.871337+0000","mapping_epoch":68,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3046","last_scrub_stamp":"2024-09-16T04:49:25.570345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:25.570345+0000","objects_scrubbed":3002,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:37.304390+0000","scrub_duration":1339,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20036,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691796+0000","last_change":"2024-09-16T05:01:59.944545+0000","last_active":"2024-09-16T05:03:09.691796+0000","last_peered":"2024-09-16T05:03:09.691796+0000","last_clean":"2024-09-16T05:03:09.691796+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:03:09.691796+0000","last_undegraded":"2024-09-16T05:03:09.691796+0000","last_fullsized":"2024-09-16T05:03:09.691796+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3345","last_scrub_stamp":"2024-09-16T04:49:33.780500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.780500+0000","objects_scrubbed":3339,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:45:06.518477+0000","scrub_duration":579,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20281,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693065+0000","last_change":"2024-09-16T04:49:28.127378+0000","last_active":"2024-09-16T05:03:09.693065+0000","last_peered":"2024-09-16T05:03:09.693065+0000","last_clean":"2024-09-16T05:03:09.693065+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:03:09.693065+0000","last_undegraded":"2024-09-16T05:03:09.693065+0000","last_fullsized":"2024-09-16T05:03:09.693065+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3083","last_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_deep_scrub":"36'3083","last_deep_scrub_stamp":"2024-09-16T04:49:28.127209+0000","last_clean_scrub_stamp":"2024-09-16T04:49:28.127209+0000","objects_scrubbed":2922,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":5,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:29.166914+0000","scrub_duration":4919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691472+0000","last_change":"2024-09-16T04:49:37.795980+0000","last_active":"2024-09-16T05:03:09.691472+0000","last_peered":"2024-09-16T05:03:09.691472+0000","last_clean":"2024-09-16T05:03:09.691472+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:03:09.691472+0000","last_undegraded":"2024-09-16T05:03:09.691472+0000","last_fullsized":"2024-09-16T05:03:09.691472+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:32:50.823432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20344,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692824+0000","last_change":"2024-09-16T05:01:29.850523+0000","last_active":"2024-09-16T05:03:09.692824+0000","last_peered":"2024-09-16T05:03:09.692824+0000","last_clean":"2024-09-16T05:03:09.692824+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:03:09.692824+0000","last_undegraded":"2024-09-16T05:03:09.692824+0000","last_fullsized":"2024-09-16T05:03:09.692824+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3289","last_scrub_stamp":"2024-09-16T04:49:32.744910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.744910+0000","objects_scrubbed":3216,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:00.957428+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053275+0000","last_change":"2024-09-16T04:48:46.454453+0000","last_active":"2024-09-16T05:03:10.053275+0000","last_peered":"2024-09-16T05:03:10.053275+0000","last_clean":"2024-09-16T05:03:10.053275+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:03:10.053275+0000","last_undegraded":"2024-09-16T05:03:10.053275+0000","last_fullsized":"2024-09-16T05:03:10.053275+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:46.765360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27338,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693866+0000","last_change":"2024-09-16T05:00:59.817107+0000","last_active":"2024-09-16T05:03:09.693866+0000","last_peered":"2024-09-16T05:03:09.693866+0000","last_clean":"2024-09-16T05:03:09.693866+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:03:09.693866+0000","last_undegraded":"2024-09-16T05:03:09.693866+0000","last_fullsized":"2024-09-16T05:03:09.693866+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3399","last_scrub_stamp":"2024-09-16T04:49:32.640105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:32.640105+0000","objects_scrubbed":3337,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:04.711258+0000","scrub_duration":1407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":148,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693578+0000","last_change":"2024-09-16T04:49:38.232856+0000","last_active":"2024-09-16T05:03:09.693578+0000","last_peered":"2024-09-16T05:03:09.693578+0000","last_clean":"2024-09-16T05:03:09.693578+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:03:09.693578+0000","last_undegraded":"2024-09-16T05:03:09.693578+0000","last_fullsized":"2024-09-16T05:03:09.693578+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:04.296483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26881,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053400+0000","last_change":"2024-09-16T05:02:41.556557+0000","last_active":"2024-09-16T05:03:10.053400+0000","last_peered":"2024-09-16T05:03:10.053400+0000","last_clean":"2024-09-16T05:03:10.053400+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:03:10.053400+0000","last_undegraded":"2024-09-16T05:03:10.053400+0000","last_fullsized":"2024-09-16T05:03:10.053400+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:02:41.556351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:02:41.556351+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:35.495820+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692638+0000","last_change":"2024-09-16T04:49:37.841481+0000","last_active":"2024-09-16T05:03:09.692638+0000","last_peered":"2024-09-16T05:03:09.692638+0000","last_clean":"2024-09-16T05:03:09.692638+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:03:09.692638+0000","last_undegraded":"2024-09-16T05:03:09.692638+0000","last_fullsized":"2024-09-16T05:03:09.692638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:55.320068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26884,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053718+0000","last_change":"2024-09-16T04:51:33.174378+0000","last_active":"2024-09-16T05:03:10.053718+0000","last_peered":"2024-09-16T05:03:10.053718+0000","last_clean":"2024-09-16T05:03:10.053718+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:03:10.053718+0000","last_undegraded":"2024-09-16T05:03:10.053718+0000","last_fullsized":"2024-09-16T05:03:10.053718+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3277","last_scrub_stamp":"2024-09-16T04:49:33.855144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.855144+0000","objects_scrubbed":3271,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:17.564355+0000","scrub_duration":447,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20025,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692648+0000","last_change":"2024-09-16T05:03:09.081130+0000","last_active":"2024-09-16T05:03:09.692648+0000","last_peered":"2024-09-16T05:03:09.692648+0000","last_clean":"2024-09-16T05:03:09.692648+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:03:09.692648+0000","last_undegraded":"2024-09-16T05:03:09.692648+0000","last_fullsized":"2024-09-16T05:03:09.692648+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19876","last_scrub_stamp":"2024-09-16T05:03:09.080970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:03:09.080970+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:19.155082+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693090+0000","last_change":"2024-09-16T04:49:37.837167+0000","last_active":"2024-09-16T05:03:09.693090+0000","last_peered":"2024-09-16T05:03:09.693090+0000","last_clean":"2024-09-16T05:03:09.693090+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:03:09.693090+0000","last_undegraded":"2024-09-16T05:03:09.693090+0000","last_fullsized":"2024-09-16T05:03:09.693090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:33:00.360958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20695,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.082246+0000","last_change":"2024-09-16T05:03:10.082306+0000","last_active":"2024-09-16T05:03:10.082246+0000","last_peered":"2024-09-16T05:03:10.082246+0000","last_clean":"2024-09-16T05:03:10.082246+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:03:10.082246+0000","last_undegraded":"2024-09-16T05:03:10.082246+0000","last_fullsized":"2024-09-16T05:03:10.082246+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20532","last_scrub_stamp":"2024-09-16T05:03:10.082042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T05:03:10.082042+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:24:17.668290+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":120,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:10.053647+0000","last_change":"2024-09-16T04:48:46.453526+0000","last_active":"2024-09-16T05:03:10.053647+0000","last_peered":"2024-09-16T05:03:10.053647+0000","last_clean":"2024-09-16T05:03:10.053647+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:03:10.053647+0000","last_undegraded":"2024-09-16T05:03:10.053647+0000","last_fullsized":"2024-09-16T05:03:10.053647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:13.760040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27041,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692471+0000","last_change":"2024-09-16T04:54:05.611427+0000","last_active":"2024-09-16T05:03:09.692471+0000","last_peered":"2024-09-16T05:03:09.692471+0000","last_clean":"2024-09-16T05:03:09.692471+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:03:09.692471+0000","last_undegraded":"2024-09-16T05:03:09.692471+0000","last_fullsized":"2024-09-16T05:03:09.692471+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'3255","last_scrub_stamp":"2024-09-16T04:49:33.024725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:33.024725+0000","objects_scrubbed":3242,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:53:18.503292+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.691799+0000","last_change":"2024-09-16T04:49:37.663717+0000","last_active":"2024-09-16T05:03:09.691799+0000","last_peered":"2024-09-16T05:03:09.691799+0000","last_clean":"2024-09-16T05:03:09.691799+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:03:09.691799+0000","last_undegraded":"2024-09-16T05:03:09.691799+0000","last_fullsized":"2024-09-16T05:03:09.691799+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:51:39.801351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.692344+0000","last_change":"2024-09-16T04:49:38.136373+0000","last_active":"2024-09-16T05:03:09.692344+0000","last_peered":"2024-09-16T05:03:09.692344+0000","last_clean":"2024-09-16T05:03:09.692344+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:03:09.692344+0000","last_undegraded":"2024-09-16T05:03:09.692344+0000","last_fullsized":"2024-09-16T05:03:09.692344+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:43.303429+0000","last_clean_scrub_stamp":"2024-09-16T04:47:43.303429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:30.006925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"51'20416","reported_seq":20602,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:09.693254+0000","last_change":"2024-09-16T04:59:57.898018+0000","last_active":"2024-09-16T05:03:09.693254+0000","last_peered":"2024-09-16T05:03:09.693254+0000","last_clean":"2024-09-16T05:03:09.693254+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:03:09.693254+0000","last_undegraded":"2024-09-16T05:03:09.693254+0000","last_fullsized":"2024-09-16T05:03:09.693254+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3269","last_scrub_stamp":"2024-09-16T04:49:27.641254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T04:47:55.367805+0000","last_clean_scrub_stamp":"2024-09-16T04:49:27.641254+0000","objects_scrubbed":3220,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:13:33.898832+0000","scrub_duration":1411,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226268,"num_write_kb":382997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30511,"num_bytes_recovered":27320320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77615104,"data_stored":77608146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986354,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415432,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93956408,"statfs":{"total":96636764160,"available":96211361792,"internally_reserved":0,"allocated":573440,"data_stored":196282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":424799913},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593892,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501516,"kb_used_data":568,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870324,"statfs":{"total":96636764160,"available":96123211776,"internally_reserved":0,"allocated":581632,"data_stored":196918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182583,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483556,"kb_used_data":77076,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93888284,"statfs":{"total":96636764160,"available":96141602816,"internally_reserved":0,"allocated":78925824,"data_stored":78541110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":77611008,"data_stored":77607529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:16.958 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-16T04:47:55.367805+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.958 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-16T04:49:26.331430+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=49, tm_sec=26, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.958 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-16T04:49:33.068296+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=49, tm_sec=33, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.958 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-16T04:47:55.367805+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=47, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.958 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-16T04:49:28.644685+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=49, tm_sec=28, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.958 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-16T04:49:25.570345+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=49, tm_sec=25, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.958 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-16T04:49:33.780500+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=49, tm_sec=33, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.959 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-16T04:49:28.127209+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=49, tm_sec=28, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.959 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T04:47:43.303429+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=47, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.959 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T04:49:32.744910+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=49, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.959 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-16T04:47:43.303429+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=47, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.959 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-16T04:49:32.640105+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=49, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.959 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-16T04:47:43.303429+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=47, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.959 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-16T05:02:41.556351+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=2, tm_sec=41, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.960 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-16T04:47:43.303429+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=47, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.960 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T04:49:33.855144+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=49, tm_sec=33, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.960 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T05:03:09.080970+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=9, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.960 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-16T04:47:43.303429+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=47, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.960 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T05:03:10.082042+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=10, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.960 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-16T04:47:43.303429+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=47, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.960 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T04:49:33.024725+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=49, tm_sec=33, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.960 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-16T04:47:43.303429+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=47, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.960 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-16T04:47:43.303429+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=47, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.961 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-16T04:49:27.641254+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=4, tm_min=49, tm_sec=27, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=5, tm_min=3, tm_sec=14, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T05:03:16.961 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T05:03:36.962 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T05:03:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-16T05:03:37.268 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-16T05:03:37.284 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":546,"stamp":"2024-09-16T05:03:36.627754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226270,"num_write_kb":382999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30512,"num_bytes_recovered":27320339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160851,"ondisk_log_size":160851,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1386092,"kb_used_data":63804,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281729428,"statfs":{"total":289910292480,"available":288490934272,"internally_reserved":0,"allocated":65335296,"data_stored":64188710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13311,"internal_metadata":1353960449},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003042"},"pg_stats":[{"pgid":"2.f","version":"51'19738","reported_seq":26564,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:19.486174+0000","last_change":"2024-09-16T05:03:19.486238+0000","last_active":"2024-09-16T05:03:19.486174+0000","last_peered":"2024-09-16T05:03:19.486174+0000","last_clean":"2024-09-16T05:03:19.486174+0000","last_became_active":"2024-09-16T04:58:54.772501+0000","last_became_peered":"2024-09-16T04:58:54.772501+0000","last_unstale":"2024-09-16T05:03:19.486174+0000","last_undegraded":"2024-09-16T05:03:19.486174+0000","last_fullsized":"2024-09-16T05:03:19.486174+0000","mapping_epoch":52,"log_start":"47'9700","ondisk_log_start":"47'9700","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19738","last_scrub_stamp":"2024-09-16T05:03:19.485988+0000","last_deep_scrub":"51'19738","last_deep_scrub_stamp":"2024-09-16T05:03:19.485988+0000","last_clean_scrub_stamp":"2024-09-16T05:03:19.485988+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:31:25.593136+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19738,"num_write_kb":39476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3342,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"51'19994","reported_seq":20168,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:20.531130+0000","last_change":"2024-09-16T05:03:20.531192+0000","last_active":"2024-09-16T05:03:20.531130+0000","last_peered":"2024-09-16T05:03:20.531130+0000","last_clean":"2024-09-16T05:03:20.531130+0000","last_became_active":"2024-09-16T04:48:46.453384+0000","last_became_peered":"2024-09-16T04:48:46.453384+0000","last_unstale":"2024-09-16T05:03:20.531130+0000","last_undegraded":"2024-09-16T05:03:20.531130+0000","last_fullsized":"2024-09-16T05:03:20.531130+0000","mapping_epoch":35,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19994","last_scrub_stamp":"2024-09-16T05:03:20.530929+0000","last_deep_scrub":"51'19994","last_deep_scrub_stamp":"2024-09-16T05:03:20.530929+0000","last_clean_scrub_stamp":"2024-09-16T05:03:20.530929+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:35.795670+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":27876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"51'19858","reported_seq":20033,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:28.232612+0000","last_change":"2024-09-16T05:03:28.232670+0000","last_active":"2024-09-16T05:03:28.232612+0000","last_peered":"2024-09-16T05:03:28.232612+0000","last_clean":"2024-09-16T05:03:28.232612+0000","last_became_active":"2024-09-16T05:02:39.743587+0000","last_became_peered":"2024-09-16T05:02:39.743587+0000","last_unstale":"2024-09-16T05:03:28.232612+0000","last_undegraded":"2024-09-16T05:03:28.232612+0000","last_fullsized":"2024-09-16T05:03:28.232612+0000","mapping_epoch":66,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19858","last_scrub_stamp":"2024-09-16T05:03:28.232414+0000","last_deep_scrub":"51'19858","last_deep_scrub_stamp":"2024-09-16T05:03:28.232414+0000","last_clean_scrub_stamp":"2024-09-16T05:03:28.232414+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:37:39.428209+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16597,"num_write_kb":26677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"51'20344","reported_seq":20472,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:18.588127+0000","last_change":"2024-09-16T05:03:18.588189+0000","last_active":"2024-09-16T05:03:18.588127+0000","last_peered":"2024-09-16T05:03:18.588127+0000","last_clean":"2024-09-16T05:03:18.588127+0000","last_became_active":"2024-09-16T04:48:46.454087+0000","last_became_peered":"2024-09-16T04:48:46.454087+0000","last_unstale":"2024-09-16T05:03:18.588127+0000","last_undegraded":"2024-09-16T05:03:18.588127+0000","last_fullsized":"2024-09-16T05:03:18.588127+0000","mapping_epoch":35,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20344","last_scrub_stamp":"2024-09-16T05:03:18.587925+0000","last_deep_scrub":"51'20344","last_deep_scrub_stamp":"2024-09-16T05:03:18.587925+0000","last_clean_scrub_stamp":"2024-09-16T05:03:18.587925+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:05.728843+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20344,"num_write_kb":40688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"51'20144","reported_seq":20421,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:16.538223+0000","last_change":"2024-09-16T05:03:16.538286+0000","last_active":"2024-09-16T05:03:16.538223+0000","last_peered":"2024-09-16T05:03:16.538223+0000","last_clean":"2024-09-16T05:03:16.538223+0000","last_became_active":"2024-09-16T04:48:45.436597+0000","last_became_peered":"2024-09-16T04:48:45.436597+0000","last_unstale":"2024-09-16T05:03:16.538223+0000","last_undegraded":"2024-09-16T05:03:16.538223+0000","last_fullsized":"2024-09-16T05:03:16.538223+0000","mapping_epoch":34,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20144","last_scrub_stamp":"2024-09-16T05:03:16.538027+0000","last_deep_scrub":"51'20144","last_deep_scrub_stamp":"2024-09-16T05:03:16.538027+0000","last_clean_scrub_stamp":"2024-09-16T05:03:16.538027+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:17.252367+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17176,"num_write_kb":28476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"51'20313","reported_seq":20476,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:29.264438+0000","last_change":"2024-09-16T05:03:29.264500+0000","last_active":"2024-09-16T05:03:29.264438+0000","last_peered":"2024-09-16T05:03:29.264438+0000","last_clean":"2024-09-16T05:03:29.264438+0000","last_became_active":"2024-09-16T05:03:09.695598+0000","last_became_peered":"2024-09-16T05:03:09.695598+0000","last_unstale":"2024-09-16T05:03:29.264438+0000","last_undegraded":"2024-09-16T05:03:29.264438+0000","last_fullsized":"2024-09-16T05:03:29.264438+0000","mapping_epoch":68,"log_start":"47'10300","ondisk_log_start":"47'10300","created":16,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20313","last_scrub_stamp":"2024-09-16T05:03:29.264249+0000","last_deep_scrub":"51'20313","last_deep_scrub_stamp":"2024-09-16T05:03:29.264249+0000","last_clean_scrub_stamp":"2024-09-16T05:03:29.264249+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:18.486376+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"51'19860","reported_seq":20045,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:27.204583+0000","last_change":"2024-09-16T05:03:27.204642+0000","last_active":"2024-09-16T05:03:27.204583+0000","last_peered":"2024-09-16T05:03:27.204583+0000","last_clean":"2024-09-16T05:03:27.204583+0000","last_became_active":"2024-09-16T05:01:59.726784+0000","last_became_peered":"2024-09-16T05:01:59.726784+0000","last_unstale":"2024-09-16T05:03:27.204583+0000","last_undegraded":"2024-09-16T05:03:27.204583+0000","last_fullsized":"2024-09-16T05:03:27.204583+0000","mapping_epoch":64,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19860","last_scrub_stamp":"2024-09-16T05:03:27.204382+0000","last_deep_scrub":"51'19860","last_deep_scrub_stamp":"2024-09-16T05:03:27.204382+0000","last_clean_scrub_stamp":"2024-09-16T05:03:27.204382+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:27.501772+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16515,"num_write_kb":26352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3401,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"51'20016","reported_seq":20290,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:17.505030+0000","last_change":"2024-09-16T05:03:17.505094+0000","last_active":"2024-09-16T05:03:17.505030+0000","last_peered":"2024-09-16T05:03:17.505030+0000","last_clean":"2024-09-16T05:03:17.505030+0000","last_became_active":"2024-09-16T04:48:45.440098+0000","last_became_peered":"2024-09-16T04:48:45.440098+0000","last_unstale":"2024-09-16T05:03:17.505030+0000","last_undegraded":"2024-09-16T05:03:17.505030+0000","last_fullsized":"2024-09-16T05:03:17.505030+0000","mapping_epoch":34,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20016","last_scrub_stamp":"2024-09-16T05:03:17.504815+0000","last_deep_scrub":"51'20016","last_deep_scrub_stamp":"2024-09-16T05:03:17.504815+0000","last_clean_scrub_stamp":"2024-09-16T05:03:17.504815+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:13.607371+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17086,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":123,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:18.099825+0000","last_change":"2024-09-16T05:03:18.099984+0000","last_active":"2024-09-16T05:03:18.099825+0000","last_peered":"2024-09-16T05:03:18.099825+0000","last_clean":"2024-09-16T05:03:18.099825+0000","last_became_active":"2024-09-16T04:49:37.795599+0000","last_became_peered":"2024-09-16T04:49:37.795599+0000","last_unstale":"2024-09-16T05:03:18.099825+0000","last_undegraded":"2024-09-16T05:03:18.099825+0000","last_fullsized":"2024-09-16T05:03:18.099825+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T05:03:18.099569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T05:03:18.099569+0000","last_clean_scrub_stamp":"2024-09-16T05:03:18.099569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:45:25.194187+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001071729,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"51'20081","reported_seq":20354,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:26.183486+0000","last_change":"2024-09-16T05:03:26.183486+0000","last_active":"2024-09-16T05:03:26.183486+0000","last_peered":"2024-09-16T05:03:26.183486+0000","last_clean":"2024-09-16T05:03:26.183486+0000","last_became_active":"2024-09-16T05:01:29.704282+0000","last_became_peered":"2024-09-16T05:01:29.704282+0000","last_unstale":"2024-09-16T05:03:26.183486+0000","last_undegraded":"2024-09-16T05:03:26.183486+0000","last_fullsized":"2024-09-16T05:03:26.183486+0000","mapping_epoch":62,"log_start":"47'10000","ondisk_log_start":"47'10000","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20081","last_scrub_stamp":"2024-09-16T05:03:26.183234+0000","last_deep_scrub":"51'20081","last_deep_scrub_stamp":"2024-09-16T05:03:26.183234+0000","last_clean_scrub_stamp":"2024-09-16T05:03:26.183234+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:15:44.977105+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16832,"num_write_kb":27232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3323,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:19.528782+0000","last_change":"2024-09-16T05:03:19.528782+0000","last_active":"2024-09-16T05:03:19.528782+0000","last_peered":"2024-09-16T05:03:19.528782+0000","last_clean":"2024-09-16T05:03:19.528782+0000","last_became_active":"2024-09-16T04:48:46.454169+0000","last_became_peered":"2024-09-16T04:48:46.454169+0000","last_unstale":"2024-09-16T05:03:19.528782+0000","last_undegraded":"2024-09-16T05:03:19.528782+0000","last_fullsized":"2024-09-16T05:03:19.528782+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T05:03:19.528382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T05:03:19.528382+0000","last_clean_scrub_stamp":"2024-09-16T05:03:19.528382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:52:58.077473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"51'20218","reported_seq":27347,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:20.457434+0000","last_change":"2024-09-16T05:03:20.457497+0000","last_active":"2024-09-16T05:03:20.457434+0000","last_peered":"2024-09-16T05:03:20.457434+0000","last_clean":"2024-09-16T05:03:20.457434+0000","last_became_active":"2024-09-16T05:00:59.671315+0000","last_became_peered":"2024-09-16T05:00:59.671315+0000","last_unstale":"2024-09-16T05:03:20.457434+0000","last_undegraded":"2024-09-16T05:03:20.457434+0000","last_fullsized":"2024-09-16T05:03:20.457434+0000","mapping_epoch":60,"log_start":"47'10200","ondisk_log_start":"47'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20218","last_scrub_stamp":"2024-09-16T05:03:20.457243+0000","last_deep_scrub":"51'20218","last_deep_scrub_stamp":"2024-09-16T05:03:20.457243+0000","last_clean_scrub_stamp":"2024-09-16T05:03:20.457243+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:39:21.016374+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16873,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"69'3","reported_seq":158,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:18.509645+0000","last_change":"2024-09-16T05:03:18.509787+0000","last_active":"2024-09-16T05:03:18.509645+0000","last_peered":"2024-09-16T05:03:18.509645+0000","last_clean":"2024-09-16T05:03:18.509645+0000","last_became_active":"2024-09-16T04:49:38.232457+0000","last_became_peered":"2024-09-16T04:49:38.232457+0000","last_unstale":"2024-09-16T05:03:18.509645+0000","last_undegraded":"2024-09-16T05:03:18.509645+0000","last_fullsized":"2024-09-16T05:03:18.509645+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'3","last_scrub_stamp":"2024-09-16T05:03:18.509419+0000","last_deep_scrub":"69'3","last_deep_scrub_stamp":"2024-09-16T05:03:18.509419+0000","last_clean_scrub_stamp":"2024-09-16T05:03:18.509419+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:09:10.134821+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0015409110000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"51'19979","reported_seq":26890,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:22.550307+0000","last_change":"2024-09-16T05:03:22.550368+0000","last_active":"2024-09-16T05:03:22.550307+0000","last_peered":"2024-09-16T05:03:22.550307+0000","last_clean":"2024-09-16T05:03:22.550307+0000","last_became_active":"2024-09-16T04:56:42.722493+0000","last_became_peered":"2024-09-16T04:56:42.722493+0000","last_unstale":"2024-09-16T05:03:22.550307+0000","last_undegraded":"2024-09-16T05:03:22.550307+0000","last_fullsized":"2024-09-16T05:03:22.550307+0000","mapping_epoch":50,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19979","last_scrub_stamp":"2024-09-16T05:03:22.550106+0000","last_deep_scrub":"51'19979","last_deep_scrub_stamp":"2024-09-16T05:03:22.550106+0000","last_clean_scrub_stamp":"2024-09-16T05:03:22.550106+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:34.555812+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"69'2","reported_seq":133,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:20.101982+0000","last_change":"2024-09-16T05:03:20.102046+0000","last_active":"2024-09-16T05:03:20.101982+0000","last_peered":"2024-09-16T05:03:20.101982+0000","last_clean":"2024-09-16T05:03:20.101982+0000","last_became_active":"2024-09-16T04:49:37.841060+0000","last_became_peered":"2024-09-16T04:49:37.841060+0000","last_unstale":"2024-09-16T05:03:20.101982+0000","last_undegraded":"2024-09-16T05:03:20.101982+0000","last_fullsized":"2024-09-16T05:03:20.101982+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'2","last_scrub_stamp":"2024-09-16T05:03:20.101787+0000","last_deep_scrub":"69'2","last_deep_scrub_stamp":"2024-09-16T05:03:20.101787+0000","last_clean_scrub_stamp":"2024-09-16T05:03:20.101787+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:31.776435+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0014480820000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"51'19978","reported_seq":26893,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:21.511052+0000","last_change":"2024-09-16T05:03:21.511113+0000","last_active":"2024-09-16T05:03:21.511052+0000","last_peered":"2024-09-16T05:03:21.511052+0000","last_clean":"2024-09-16T05:03:21.511052+0000","last_became_active":"2024-09-16T04:51:33.161891+0000","last_became_peered":"2024-09-16T04:51:33.161891+0000","last_unstale":"2024-09-16T05:03:21.511052+0000","last_undegraded":"2024-09-16T05:03:21.511052+0000","last_fullsized":"2024-09-16T05:03:21.511052+0000","mapping_epoch":46,"log_start":"47'9900","ondisk_log_start":"47'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19978","last_scrub_stamp":"2024-09-16T05:03:21.510855+0000","last_deep_scrub":"51'19978","last_deep_scrub_stamp":"2024-09-16T05:03:21.510855+0000","last_clean_scrub_stamp":"2024-09-16T05:03:21.510855+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:26.488513+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":26860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3346,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"51'19876","reported_seq":20034,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:25.204048+0000","last_change":"2024-09-16T05:03:25.204110+0000","last_active":"2024-09-16T05:03:25.204048+0000","last_peered":"2024-09-16T05:03:25.204048+0000","last_clean":"2024-09-16T05:03:25.204048+0000","last_became_active":"2024-09-16T05:00:26.669798+0000","last_became_peered":"2024-09-16T05:00:26.669798+0000","last_unstale":"2024-09-16T05:03:25.204048+0000","last_undegraded":"2024-09-16T05:03:25.204048+0000","last_fullsized":"2024-09-16T05:03:25.204048+0000","mapping_epoch":58,"log_start":"47'9800","ondisk_log_start":"47'9800","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'19876","last_scrub_stamp":"2024-09-16T05:03:25.203802+0000","last_deep_scrub":"51'19876","last_deep_scrub_stamp":"2024-09-16T05:03:25.203802+0000","last_clean_scrub_stamp":"2024-09-16T05:03:25.203802+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:17.270393+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":123,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:19.146273+0000","last_change":"2024-09-16T05:03:19.146375+0000","last_active":"2024-09-16T05:03:19.146273+0000","last_peered":"2024-09-16T05:03:19.146273+0000","last_clean":"2024-09-16T05:03:19.146273+0000","last_became_active":"2024-09-16T04:49:37.836796+0000","last_became_peered":"2024-09-16T04:49:37.836796+0000","last_unstale":"2024-09-16T05:03:19.146273+0000","last_undegraded":"2024-09-16T05:03:19.146273+0000","last_fullsized":"2024-09-16T05:03:19.146273+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T05:03:19.146018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T05:03:19.146018+0000","last_clean_scrub_stamp":"2024-09-16T05:03:19.146018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:24.519448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290266,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"51'20532","reported_seq":20704,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:23.117097+0000","last_change":"2024-09-16T05:03:23.117156+0000","last_active":"2024-09-16T05:03:23.117097+0000","last_peered":"2024-09-16T05:03:23.117097+0000","last_clean":"2024-09-16T05:03:23.117097+0000","last_became_active":"2024-09-16T04:59:25.730096+0000","last_became_peered":"2024-09-16T04:59:25.730096+0000","last_unstale":"2024-09-16T05:03:23.117097+0000","last_undegraded":"2024-09-16T05:03:23.117097+0000","last_fullsized":"2024-09-16T05:03:23.117097+0000","mapping_epoch":54,"log_start":"47'10500","ondisk_log_start":"47'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20532","last_scrub_stamp":"2024-09-16T05:03:23.116903+0000","last_deep_scrub":"51'20532","last_deep_scrub_stamp":"2024-09-16T05:03:23.116903+0000","last_clean_scrub_stamp":"2024-09-16T05:03:23.116903+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:36:35.956711+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":130,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:17.537332+0000","last_change":"2024-09-16T05:03:17.537332+0000","last_active":"2024-09-16T05:03:17.537332+0000","last_peered":"2024-09-16T05:03:17.537332+0000","last_clean":"2024-09-16T05:03:17.537332+0000","last_became_active":"2024-09-16T04:48:46.453223+0000","last_became_peered":"2024-09-16T04:48:46.453223+0000","last_unstale":"2024-09-16T05:03:17.537332+0000","last_undegraded":"2024-09-16T05:03:17.537332+0000","last_fullsized":"2024-09-16T05:03:17.537332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T05:03:17.537115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T05:03:17.537115+0000","last_clean_scrub_stamp":"2024-09-16T05:03:17.537115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:23:11.961727+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"51'20198","reported_seq":27050,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:22.114867+0000","last_change":"2024-09-16T05:03:22.114982+0000","last_active":"2024-09-16T05:03:22.114867+0000","last_peered":"2024-09-16T05:03:22.114867+0000","last_clean":"2024-09-16T05:03:22.114867+0000","last_became_active":"2024-09-16T04:54:05.567265+0000","last_became_peered":"2024-09-16T04:54:05.567265+0000","last_unstale":"2024-09-16T05:03:22.114867+0000","last_undegraded":"2024-09-16T05:03:22.114867+0000","last_fullsized":"2024-09-16T05:03:22.114867+0000","mapping_epoch":48,"log_start":"47'10100","ondisk_log_start":"47'10100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'20198","last_scrub_stamp":"2024-09-16T05:03:22.114625+0000","last_deep_scrub":"51'20198","last_deep_scrub_stamp":"2024-09-16T05:03:22.114625+0000","last_clean_scrub_stamp":"2024-09-16T05:03:22.114625+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:35:13.752293+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16952,"num_write_kb":27420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":123,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:17.068664+0000","last_change":"2024-09-16T05:03:17.068714+0000","last_active":"2024-09-16T05:03:17.068664+0000","last_peered":"2024-09-16T05:03:17.068664+0000","last_clean":"2024-09-16T05:03:17.068664+0000","last_became_active":"2024-09-16T04:49:37.662665+0000","last_became_peered":"2024-09-16T04:49:37.662665+0000","last_unstale":"2024-09-16T05:03:17.068664+0000","last_undegraded":"2024-09-16T05:03:17.068664+0000","last_fullsized":"2024-09-16T05:03:17.068664+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T05:03:17.068426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T05:03:17.068426+0000","last_clean_scrub_stamp":"2024-09-16T05:03:17.068426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:50.805154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096277199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":127,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:21.061290+0000","last_change":"2024-09-16T05:03:21.061346+0000","last_active":"2024-09-16T05:03:21.061290+0000","last_peered":"2024-09-16T05:03:21.061290+0000","last_clean":"2024-09-16T05:03:21.061290+0000","last_became_active":"2024-09-16T04:49:38.135595+0000","last_became_peered":"2024-09-16T04:49:38.135595+0000","last_unstale":"2024-09-16T05:03:21.061290+0000","last_undegraded":"2024-09-16T05:03:21.061290+0000","last_fullsized":"2024-09-16T05:03:21.061290+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T05:03:21.061060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T05:03:21.061060+0000","last_clean_scrub_stamp":"2024-09-16T05:03:21.061060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:57.156551+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00125067,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"69'20417","reported_seq":20612,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T05:03:24.178587+0000","last_change":"2024-09-16T05:03:24.178655+0000","last_active":"2024-09-16T05:03:24.178587+0000","last_peered":"2024-09-16T05:03:24.178587+0000","last_clean":"2024-09-16T05:03:24.178587+0000","last_became_active":"2024-09-16T04:59:57.716705+0000","last_became_peered":"2024-09-16T04:59:57.716705+0000","last_unstale":"2024-09-16T05:03:24.178587+0000","last_undegraded":"2024-09-16T05:03:24.178587+0000","last_fullsized":"2024-09-16T05:03:24.178587+0000","mapping_epoch":56,"log_start":"47'10400","ondisk_log_start":"47'10400","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"69'20417","last_scrub_stamp":"2024-09-16T05:03:24.178409+0000","last_deep_scrub":"69'20417","last_deep_scrub_stamp":"2024-09-16T05:03:24.178409+0000","last_clean_scrub_stamp":"2024-09-16T05:03:24.178409+0000","objects_scrubbed":1,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:54.574171+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17185,"num_write_kb":27932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":226268,"num_write_kb":382997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30511,"num_bytes_recovered":27320320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62869504,"data_stored":62862546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160846,"ondisk_log_size":160846,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986358,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415424,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93956416,"statfs":{"total":96636764160,"available":96211369984,"internally_reserved":0,"allocated":573440,"data_stored":196282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":424799913},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":43,"seq":184683593896,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501512,"kb_used_data":568,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870328,"statfs":{"total":96636764160,"available":96123215872,"internally_reserved":0,"allocated":581632,"data_stored":196918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":512945838},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182587,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469156,"kb_used_data":62676,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93902684,"statfs":{"total":96636764160,"available":96156348416,"internally_reserved":0,"allocated":64180224,"data_stored":63795510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":62865408,"data_stored":62861929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T05:03:37.286 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T05:03:37.609 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T05:03:37.610 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T05:03:37.610 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T05:03:37.610 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T05:03:43.453 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-16T05:03:43.711 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T05:03:43.712 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T05:03:43.712 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T05:03:48.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-16T05:03:48.957 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-16T05:03:49.814 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T05:03:49.814 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T05:03:49.814 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T05:03:54.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-16T05:03:54.361 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-16T05:03:55.916 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T05:03:55.916 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T05:03:55.917 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T05:03:55.917 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T05:03:55.947 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T05:03:55.947 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T05:03:55.947 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T05:03:55.947 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T05:03:56.006 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T05:03:56.006 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T05:03:56.007 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T05:03:56.070 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T05:03:56.071 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T05:03:56.071 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T05:03:56.133 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T05:03:56.133 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T05:03:56.133 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-16T05:03:56.224 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-16T05:03:56.224 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T05:03:57.329 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-16T05:03:57.330 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T05:03:57.575 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-16T05:03:57.576 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T05:03:57.909 INFO:tasks.ceph:Archiving mon data... 2024-09-16T05:03:57.910 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_21:00:02-rados-squid-distro-default-smithi/7906051/data/mon.a.tgz 2024-09-16T05:03:57.910 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-09-16T05:03:57.929 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.PwbiOyDJfu 2024-09-16T05:03:57.930 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.PwbiOyDJfu 2024-09-16T05:03:58.061 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.PwbiOyDJfu 2024-09-16T05:03:58.150 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.PwbiOyDJfu is 299KB 2024-09-16T05:03:58.212 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.PwbiOyDJfu 2024-09-16T05:03:58.233 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_21:00:02-rados-squid-distro-default-smithi/7906051/data/mon.b.tgz 2024-09-16T05:03:58.234 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-09-16T05:03:58.282 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.n2QssvYvbW 2024-09-16T05:03:58.283 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.n2QssvYvbW 2024-09-16T05:03:58.399 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.n2QssvYvbW 2024-09-16T05:03:58.463 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.n2QssvYvbW is 299KB 2024-09-16T05:03:58.524 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.n2QssvYvbW 2024-09-16T05:03:58.529 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_21:00:02-rados-squid-distro-default-smithi/7906051/data/mon.c.tgz 2024-09-16T05:03:58.529 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-09-16T05:03:58.578 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.tesxgs3sRL 2024-09-16T05:03:58.579 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.tesxgs3sRL 2024-09-16T05:03:58.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.tesxgs3sRL 2024-09-16T05:03:58.754 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.tesxgs3sRL is 295KB 2024-09-16T05:03:58.811 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.tesxgs3sRL 2024-09-16T05:03:58.816 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T05:03:58.816 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T05:03:58.926 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T05:03:58.926 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T05:03:58.928 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:00:02-rados-squid-distro-default-smithi/7906051/remote/smithi012/crash 2024-09-16T05:03:58.929 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T05:03:58.983 INFO:tasks.ceph:Compressing logs... 2024-09-16T05:03:58.983 DEBUG:teuthology.orchestra.run.smithi012:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T05:03:59.052 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43958.log 2024-09-16T05:03:59.053 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44506.log 2024-09-16T05:03:59.053 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43958.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43958.log.gz 2024-09-16T05:03:59.053 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35988.log 2024-09-16T05:03:59.056 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49921.log 2024-09-16T05:03:59.056 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44506.log.gz 2024-09-16T05:03:59.056 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35988.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.35988.log.gz 2024-09-16T05:03:59.056 INFO:teuthology.orchestra.run.smithi012.stderr: /var/log/ceph/ceph-client.admin.40103.log 2024-09-16T05:03:59.056 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30998.log 2024-09-16T05:03:59.056 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49921.log.gz 2024-09-16T05:03:59.056 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40103.log.gz 2024-09-16T05:03:59.057 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41507.log 2024-09-16T05:03:59.057 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38280.log 2024-09-16T05:03:59.057 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30998.log.gz 2024-09-16T05:03:59.057 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41507.log.gz 2024-09-16T05:03:59.057 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47891.log 2024-09-16T05:03:59.063 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38644.log 2024-09-16T05:03:59.063 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38280.log.gz 2024-09-16T05:03:59.063 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38774.log 2024-09-16T05:03:59.063 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47891.log.gz 2024-09-16T05:03:59.063 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35441.log 2024-09-16T05:03:59.063 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38644.log.gz 2024-09-16T05:03:59.063 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51694.log 2024-09-16T05:03:59.063 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38774.log.gz 2024-09-16T05:03:59.063 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46460.log 2024-09-16T05:03:59.063 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35441.log.gz 2024-09-16T05:03:59.064 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51694.log.gz 2024-09-16T05:03:59.064 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41299.log 2024-09-16T05:03:59.064 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46381.log 2024-09-16T05:03:59.064 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46460.log.gz 2024-09-16T05:03:59.064 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30607.log 2024-09-16T05:03:59.064 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41299.log.gz 2024-09-16T05:03:59.064 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46381.log.gz 2024-09-16T05:03:59.064 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50312.log 2024-09-16T05:03:59.064 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44688.log 2024-09-16T05:03:59.064 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30607.log.gz 2024-09-16T05:03:59.064 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48333.log 2024-09-16T05:03:59.064 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50312.log.gz 2024-09-16T05:03:59.065 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37655.log 2024-09-16T05:03:59.065 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44688.log.gz 2024-09-16T05:03:59.065 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42106.log 2024-09-16T05:03:59.065 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48333.log.gz 2024-09-16T05:03:59.065 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51226.log 2024-09-16T05:03:59.065 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37655.log.gz 2024-09-16T05:03:59.065 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42106.log.gz 2024-09-16T05:03:59.065 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25091.log 2024-09-16T05:03:59.065 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44792.log 2024-09-16T05:03:59.065 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51226.log.gz 2024-09-16T05:03:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.25091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48307.log 2024-09-16T05:03:59.072 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25091.log.gz 2024-09-16T05:03:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44792.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44792.log.gz -5 2024-09-16T05:03:59.072 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34091.log 2024-09-16T05:03:59.072 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39216.log 2024-09-16T05:03:59.073 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48307.log.gz 2024-09-16T05:03:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51200.log 2024-09-16T05:03:59.073 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34091.log.gz 2024-09-16T05:03:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40675.log 2024-09-16T05:03:59.073 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39216.log.gz 2024-09-16T05:03:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38358.log 2024-09-16T05:03:59.073 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51200.log.gz 2024-09-16T05:03:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46434.log 2024-09-16T05:03:59.073 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40675.log.gz 2024-09-16T05:03:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38358.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38358.log.gz 2024-09-16T05:03:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43672.log 2024-09-16T05:03:59.073 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44558.log 2024-09-16T05:03:59.073 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46434.log.gz 2024-09-16T05:03:59.074 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30088.log 2024-09-16T05:03:59.074 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43672.log.gz 2024-09-16T05:03:59.074 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44376.log 2024-09-16T05:03:59.074 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44558.log.gz 2024-09-16T05:03:59.074 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40831.log 2024-09-16T05:03:59.074 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30088.log.gz 2024-09-16T05:03:59.074 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30377.log 2024-09-16T05:03:59.074 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44376.log.gz 2024-09-16T05:03:59.074 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50990.log 2024-09-16T05:03:59.074 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40831.log.gz 2024-09-16T05:03:59.074 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47474.log 2024-09-16T05:03:59.074 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30377.log.gz 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51460.log 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50990.log.gz 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47474.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47474.log.gz 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47579.log 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38410.log 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51460.log.gz 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38072.log 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47579.log.gz 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41195.log 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38410.log.gz 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37681.log 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38072.log.gz 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46225.log 2024-09-16T05:03:59.075 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41195.log.gz 2024-09-16T05:03:59.086 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31754.log 2024-09-16T05:03:59.086 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37681.log.gz 2024-09-16T05:03:59.086 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45341.log 2024-09-16T05:03:59.086 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46225.log.gz 2024-09-16T05:03:59.086 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44532.log 2024-09-16T05:03:59.086 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31754.log.gz 2024-09-16T05:03:59.086 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37213.log 2024-09-16T05:03:59.086 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45341.log.gz 2024-09-16T05:03:59.086 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36296.log 2024-09-16T05:03:59.087 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44532.log.gz 2024-09-16T05:03:59.087 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35944.log 2024-09-16T05:03:59.087 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37213.log.gz 2024-09-16T05:03:59.087 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50521.log 2024-09-16T05:03:59.087 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36296.log.gz 2024-09-16T05:03:59.087 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41846.log 2024-09-16T05:03:59.087 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35944.log.gz 2024-09-16T05:03:59.087 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49791.log 2024-09-16T05:03:59.087 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50521.log.gz 2024-09-16T05:03:59.087 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39268.log 2024-09-16T05:03:59.087 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41846.log.gz 2024-09-16T05:03:59.087 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50078.log 2024-09-16T05:03:59.087 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49791.log.gz 2024-09-16T05:03:59.088 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42863.log 2024-09-16T05:03:59.088 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39268.log.gz 2024-09-16T05:03:59.088 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50078.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50078.log.gz 2024-09-16T05:03:59.088 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45471.log 2024-09-16T05:03:59.088 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46590.log 2024-09-16T05:03:59.088 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42863.log.gz 2024-09-16T05:03:59.088 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35181.log 2024-09-16T05:03:59.088 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45471.log.gz 2024-09-16T05:03:59.088 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46590.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46590.log.gz 2024-09-16T05:03:59.088 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38098.log 2024-09-16T05:03:59.088 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41143.log 2024-09-16T05:03:59.088 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35181.log.gz 2024-09-16T05:03:59.088 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38098.log.gz 2024-09-16T05:03:59.089 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37603.log 2024-09-16T05:03:59.089 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33047.log 2024-09-16T05:03:59.089 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41143.log.gz 2024-09-16T05:03:59.089 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51642.log 2024-09-16T05:03:59.089 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37603.log.gz 2024-09-16T05:03:59.089 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38436.log 2024-09-16T05:03:59.089 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33047.log.gz 2024-09-16T05:03:59.089 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47631.log 2024-09-16T05:03:59.089 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51642.log.gz 2024-09-16T05:03:59.089 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49271.log 2024-09-16T05:03:59.089 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38436.log.gz 2024-09-16T05:03:59.089 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28055.log 2024-09-16T05:03:59.089 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47631.log.gz 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37525.log 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49271.log.gz 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.28055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40753.log 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28055.log.gz 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37525.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37525.log.gz 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39739.log 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45627.log 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40753.log.gz 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46876.log 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39739.log.gz 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45627.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.45627.log.gz 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr: /var/log/ceph/ceph-client.admin.47943.log 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37369.log 2024-09-16T05:03:59.090 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46876.log.gz 2024-09-16T05:03:59.091 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50260.log 2024-09-16T05:03:59.091 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47943.log.gz 2024-09-16T05:03:59.091 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37369.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37369.log.gz 2024-09-16T05:03:59.091 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38800.log 2024-09-16T05:03:59.109 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48177.log 2024-09-16T05:03:59.109 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50260.log.gz 2024-09-16T05:03:59.109 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46980.log 2024-09-16T05:03:59.109 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38800.log.gz 2024-09-16T05:03:59.109 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48177.log.gz 2024-09-16T05:03:59.109 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-16T05:03:59.109 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34687.log 2024-09-16T05:03:59.110 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46980.log.gz 2024-09-16T05:03:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39895.log 2024-09-16T05:03:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37942.log 2024-09-16T05:03:59.110 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34687.log.gz 2024-09-16T05:03:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45315.log 2024-09-16T05:03:59.110 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39895.log.gz 2024-09-16T05:03:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35103.log 2024-09-16T05:03:59.110 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37942.log.gz 2024-09-16T05:03:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30734.log 2024-09-16T05:03:59.110 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45315.log.gz 2024-09-16T05:03:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39008.log 2024-09-16T05:03:59.111 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35103.log.gz 2024-09-16T05:03:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30734.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30734.log.gz 2024-09-16T05:03:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30018.log 2024-09-16T05:03:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39008.log.gz 2024-09-16T05:03:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49635.log 2024-09-16T05:03:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31600.log 2024-09-16T05:03:59.111 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30018.log.gz 2024-09-16T05:03:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36120.log 2024-09-16T05:03:59.111 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49635.log.gz 2024-09-16T05:03:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31600.log.gz 2024-09-16T05:03:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35389.log 2024-09-16T05:03:59.111 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40233.log 2024-09-16T05:03:59.112 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36120.log.gz 2024-09-16T05:03:59.112 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35389.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35389.log.gz 2024-09-16T05:03:59.112 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49141.log 2024-09-16T05:03:59.112 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40233.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.40233.log.gz 2024-09-16T05:03:59.112 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.29516.log 2024-09-16T05:03:59.112 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38176.log 2024-09-16T05:03:59.112 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49141.log.gz 2024-09-16T05:03:59.112 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44114.log 0.0% 2024-09-16T05:03:59.112 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29516.log.gz 2024-09-16T05:03:59.112 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38176.log.gz 2024-09-16T05:03:59.112 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37421.log 2024-09-16T05:03:59.112 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30659.log 2024-09-16T05:03:59.113 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44114.log.gz 2024-09-16T05:03:59.113 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31380.log 2024-09-16T05:03:59.113 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37421.log.gz 2024-09-16T05:03:59.113 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35701.log 2024-09-16T05:03:59.113 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30659.log.gz 2024-09-16T05:03:59.113 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46642.log 2024-09-16T05:03:59.113 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31380.log.gz 2024-09-16T05:03:59.113 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30581.log 2024-09-16T05:03:59.113 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35701.log.gz 2024-09-16T05:03:59.113 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41273.log 2024-09-16T05:03:59.113 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46642.log.gz 2024-09-16T05:03:59.113 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48984.log 2024-09-16T05:03:59.113 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30581.log.gz 2024-09-16T05:03:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41273.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41273.log.gz 2024-09-16T05:03:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42499.log 2024-09-16T05:03:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29726.log 2024-09-16T05:03:59.114 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48984.log.gz 2024-09-16T05:03:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42499.log.gz 2024-09-16T05:03:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30481.log 2024-09-16T05:03:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44454.log 2024-09-16T05:03:59.114 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29726.log.gz 2024-09-16T05:03:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30481.log.gz 2024-09-16T05:03:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47501.log 2024-09-16T05:03:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49479.log 2024-09-16T05:03:59.114 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44454.log.gz 2024-09-16T05:03:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.47501.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35519.log 2024-09-16T05:03:59.115 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47501.log.gz 2024-09-16T05:03:59.115 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34563.log 2024-09-16T05:03:59.115 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49479.log.gz 2024-09-16T05:03:59.115 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40727.log 2024-09-16T05:03:59.115 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35519.log.gz 2024-09-16T05:03:59.115 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50860.log 2024-09-16T05:03:59.115 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34563.log.gz 2024-09-16T05:03:59.115 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47709.log 2024-09-16T05:03:59.115 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40727.log.gz 2024-09-16T05:03:59.115 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39791.log 2024-09-16T05:03:59.115 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50860.log.gz 2024-09-16T05:03:59.115 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51980.log 2024-09-16T05:03:59.115 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47709.log.gz 2024-09-16T05:03:59.116 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43776.log 2024-09-16T05:03:59.116 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39791.log.gz 2024-09-16T05:03:59.116 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47084.log 2024-09-16T05:03:59.116 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51980.log.gz 2024-09-16T05:03:59.116 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43776.log: /var/log/ceph/ceph-client.admin.46668.log 2024-09-16T05:03:59.116 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43776.log.gz 2024-09-16T05:03:59.116 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45783.log 2024-09-16T05:03:59.116 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47084.log.gz 2024-09-16T05:03:59.116 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45601.log 2024-09-16T05:03:59.116 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46668.log.gz 2024-09-16T05:03:59.116 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41690.log/var/log/ceph/ceph-client.admin.45783.log: 2024-09-16T05:03:59.116 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45783.log.gz 2024-09-16T05:03:59.116 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46121.log 2024-09-16T05:03:59.117 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45601.log.gz 2024-09-16T05:03:59.117 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29548.log 2024-09-16T05:03:59.117 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41690.log.gz 2024-09-16T05:03:59.117 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51408.log/var/log/ceph/ceph-client.admin.46121.log: 2024-09-16T05:03:59.117 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46121.log.gz 2024-09-16T05:03:59.117 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39687.log 2024-09-16T05:03:59.117 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29548.log.gz 2024-09-16T05:03:59.117 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46928.log 2024-09-16T05:03:59.117 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51408.log.gz 2024-09-16T05:03:59.117 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40935.log 2024-09-16T05:03:59.117 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39687.log.gz 2024-09-16T05:03:59.117 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35523.log 2024-09-16T05:03:59.118 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46928.log.gz 2024-09-16T05:03:59.118 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25237.log 2024-09-16T05:03:59.118 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40935.log.gz 2024-09-16T05:03:59.118 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.35523.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50208.log 2024-09-16T05:03:59.118 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35523.log.gz 2024-09-16T05:03:59.118 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.25237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32888.log 2024-09-16T05:03:59.118 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25237.log.gz 2024-09-16T05:03:59.118 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39633.log 2024-09-16T05:03:59.118 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50208.log.gz 2024-09-16T05:03:59.179 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45835.log 2024-09-16T05:03:59.180 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32888.log.gz 2024-09-16T05:03:59.180 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42993.log 2024-09-16T05:03:59.180 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39633.log.gz 2024-09-16T05:03:59.180 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44010.log 2024-09-16T05:03:59.180 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45835.log.gz 2024-09-16T05:03:59.180 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45367.log 2024-09-16T05:03:59.180 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42993.log.gz 2024-09-16T05:03:59.180 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35623.log 2024-09-16T05:03:59.180 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44010.log.gz 2024-09-16T05:03:59.180 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29490.log 2024-09-16T05:03:59.180 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45367.log.gz 2024-09-16T05:03:59.180 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49089.log 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35623.log.gz 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29776.log 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29490.log.gz 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43802.log 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49089.log.gz 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32995.log 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29776.log.gz 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44298.log 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43802.log.gz 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32995.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32995.log.gz 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47657.log 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47787.log 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44298.log.gz 2024-09-16T05:03:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47657.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47657.log.gz 2024-09-16T05:03:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35285.log 2024-09-16T05:03:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35337.log 2024-09-16T05:03:59.182 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47787.log.gz 2024-09-16T05:03:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.35285.log: /var/log/ceph/ceph-client.admin.31710.log 2024-09-16T05:03:59.182 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35285.log.gz 2024-09-16T05:03:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36321.log 2024-09-16T05:03:59.182 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35337.log.gz 2024-09-16T05:03:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49375.log 2024-09-16T05:03:59.182 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31710.log.gz 2024-09-16T05:03:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35415.log 2024-09-16T05:03:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36321.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36321.log.gz 2024-09-16T05:03:59.182 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42837.log 2024-09-16T05:03:59.182 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49375.log.gz 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35415.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.35415.log.gz --verbose 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr: -- /var/log/ceph/ceph-client.admin.33043.log 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37291.log 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42837.log.gz 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31820.log 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33043.log.gz 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39346.log 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37291.log.gz 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42941.log 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31820.log.gz 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38722.log 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39346.log.gz 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44270.log 2024-09-16T05:03:59.183 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42941.log.gz 2024-09-16T05:03:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35233.log 2024-09-16T05:03:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38722.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38722.log.gz 2024-09-16T05:03:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48698.log 2024-09-16T05:03:59.184 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44270.log.gz 2024-09-16T05:03:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37317.log 2024-09-16T05:03:59.184 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35233.log.gz 2024-09-16T05:03:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48698.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-16T05:03:59.184 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48698.log.gz 2024-09-16T05:03:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44166.log 2024-09-16T05:03:59.184 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37317.log.gz 2024-09-16T05:03:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37968.log 2024-09-16T05:03:59.184 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47396.log 2024-09-16T05:03:59.184 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44166.log.gz 2024-09-16T05:03:59.185 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.37968.log: /var/log/ceph/ceph-client.admin.43204.log 2024-09-16T05:03:59.185 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37968.log.gz 2024-09-16T05:03:59.185 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50130.log 2024-09-16T05:03:59.185 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47396.log.gz 2024-09-16T05:03:59.185 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46772.log 2024-09-16T05:03:59.185 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43204.log.gz 2024-09-16T05:03:59.185 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39765.log 2024-09-16T05:03:59.185 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50130.log.gz 2024-09-16T05:03:59.185 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34611.log 2024-09-16T05:03:59.185 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46772.log.gz 2024-09-16T05:03:59.185 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34869.log 2024-09-16T05:03:59.185 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39765.log.gz 2024-09-16T05:03:59.185 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.34611.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51434.log 2024-09-16T05:03:59.186 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34611.log.gz 2024-09-16T05:03:59.186 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44740.log 2024-09-16T05:03:59.186 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34869.log.gz 2024-09-16T05:03:59.186 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41924.log 2024-09-16T05:03:59.186 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51434.log.gz 2024-09-16T05:03:59.186 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48203.log 2024-09-16T05:03:59.186 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44740.log.gz 2024-09-16T05:03:59.186 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31042.log 2024-09-16T05:03:59.186 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41924.log.gz 2024-09-16T05:03:59.186 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48203.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48203.log.gz 2024-09-16T05:03:59.186 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50938.log 2024-09-16T05:03:59.186 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34459.log 2024-09-16T05:03:59.186 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31042.log.gz 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42317.log 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50938.log.gz 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34459.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34459.log.gz 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30888.log 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50808.log 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42317.log.gz 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30888.log.gz 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41742.log 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50808.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50808.log.gz 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33199.log 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51824.log 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41742.log.gz 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33199.log.gz 2024-09-16T05:03:59.187 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31732.log 2024-09-16T05:03:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48411.log 2024-09-16T05:03:59.188 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51824.log.gz 2024-09-16T05:03:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31732.log.gz 2024-09-16T05:03:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43464.log 2024-09-16T05:03:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46251.log 2024-09-16T05:03:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48411.log: /var/log/ceph/ceph-client.admin.43464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47058.log 0.0% 2024-09-16T05:03:59.188 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48411.log.gz 2024-09-16T05:03:59.188 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43464.log.gz 2024-09-16T05:03:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46251.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46251.log.gz 2024-09-16T05:03:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38202.log 2024-09-16T05:03:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48255.log 2024-09-16T05:03:59.188 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47058.log.gz 2024-09-16T05:03:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38202.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38202.log.gz -5 2024-09-16T05:03:59.189 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34429.log 2024-09-16T05:03:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48255.log.gz 2024-09-16T05:03:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37577.log 2024-09-16T05:03:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39529.log 2024-09-16T05:03:59.189 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34429.log.gz 2024-09-16T05:03:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37577.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.36252.log -- replaced with /var/log/ceph/ceph-client.admin.37577.log.gz 2024-09-16T05:03:59.189 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-09-16T05:03:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.39529.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40779.log 2024-09-16T05:03:59.189 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39529.log.gz 2024-09-16T05:03:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36252.log.gz 2024-09-16T05:03:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35051.log 2024-09-16T05:03:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47761.log 2024-09-16T05:03:59.189 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40779.log.gz 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35051.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35051.log.gz 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49010.log 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49037.log 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47761.log.gz 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49010.log.gz 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30244.log 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30296.log 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49037.log.gz 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30244.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30244.log.gz 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41403.log 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42473.log 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30296.log.gz 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41403.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41403.log.gz 2024-09-16T05:03:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50416.log 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34247.log 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42473.log.gz 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50416.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50416.log.gz 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31284.log 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34247.log: /var/log/ceph/ceph-mon.a.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34247.log.gz 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40337.log 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50677.log 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31284.log.gz 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43282.log 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40337.log.gz 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43071.log 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50677.log.gz 2024-09-16T05:03:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44062.log 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43282.log.gz 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43071.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43071.log.gz 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49687.log 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30932.log 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44062.log.gz 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45235.log 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49687.log.gz 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35812.log 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30932.log.gz 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45235.log.gz 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41013.log 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51538.log 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35812.log.gz 2024-09-16T05:03:59.192 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41013.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41013.log.gz 2024-09-16T05:03:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44871.log 2024-09-16T05:03:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46017.log 2024-09-16T05:03:59.193 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51538.log.gz 2024-09-16T05:03:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44871.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44871.log.gz -5 2024-09-16T05:03:59.193 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47448.log 2024-09-16T05:03:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30095.log 2024-09-16T05:03:59.193 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46017.log.gz 2024-09-16T05:03:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49895.log 2024-09-16T05:03:59.193 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47448.log.gz 2024-09-16T05:03:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30095.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.30095.log.gz 2024-09-16T05:03:59.193 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46486.log 2024-09-16T05:03:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39581.log 2024-09-16T05:03:59.194 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49895.log.gz 2024-09-16T05:03:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47344.log 2024-09-16T05:03:59.194 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46486.log.gz 2024-09-16T05:03:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39581.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39581.log.gz 2024-09-16T05:03:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41976.log 2024-09-16T05:03:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34973.log 2024-09-16T05:03:59.194 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47344.log.gz 2024-09-16T05:03:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41976.log.gz 2024-09-16T05:03:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50234.log 2024-09-16T05:03:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50468.log 2024-09-16T05:03:59.194 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34973.log.gz 2024-09-16T05:03:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50912.log 2024-09-16T05:03:59.194 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50234.log.gz 2024-09-16T05:03:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30166.log 2024-09-16T05:03:59.195 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50468.log.gz 2024-09-16T05:03:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50912.log.gz 2024-09-16T05:03:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41950.log 2024-09-16T05:03:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40649.log 2024-09-16T05:03:59.195 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30166.log.gz 2024-09-16T05:03:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41950.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.30477.log 2024-09-16T05:03:59.195 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41950.log.gz 2024-09-16T05:03:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36076.log 2024-09-16T05:03:59.196 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40649.log.gz 2024-09-16T05:03:59.196 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42629.log 2024-09-16T05:03:59.196 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30477.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30477.log.gz 2024-09-16T05:03:59.196 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36076.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36076.log.gz 2024-09-16T05:03:59.196 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40571.log 2024-09-16T05:03:59.196 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41065.log 2024-09-16T05:03:59.196 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42629.log.gz 2024-09-16T05:03:59.196 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40571.log.gz 2024-09-16T05:03:59.196 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40129.log 2024-09-16T05:03:59.196 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31798.log 2024-09-16T05:03:59.196 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41065.log.gz 2024-09-16T05:03:59.196 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30822.log 2024-09-16T05:03:59.197 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40129.log.gz 2024-09-16T05:03:59.197 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31798.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31798.log.gz 2024-09-16T05:03:59.197 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49609.log 2024-09-16T05:03:59.197 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25316.log 2024-09-16T05:03:59.197 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30822.log.gz 2024-09-16T05:03:59.197 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36032.log 2024-09-16T05:03:59.197 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49609.log.gz 2024-09-16T05:03:59.197 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.25316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40259.log 2024-09-16T05:03:59.197 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25316.log.gz 2024-09-16T05:03:59.197 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36032.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.50390.log -- replaced with /var/log/ceph/ceph-client.admin.36032.log.gz 2024-09-16T05:03:59.197 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-09-16T05:03:59.197 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45131.log 2024-09-16T05:03:59.197 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40259.log.gz 2024-09-16T05:03:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50390.log.gz 2024-09-16T05:03:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52006.log 2024-09-16T05:03:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45001.log 2024-09-16T05:03:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45131.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45131.log.gz 2024-09-16T05:03:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.52006.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46746.log 2024-09-16T05:03:59.198 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52006.log.gz 2024-09-16T05:03:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35649.log 2024-09-16T05:03:59.198 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45001.log.gz 2024-09-16T05:03:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42915.log 2024-09-16T05:03:59.198 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46746.log.gz 2024-09-16T05:03:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35649.log.gz 2024-09-16T05:03:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29854.log 2024-09-16T05:03:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37863.log 2024-09-16T05:03:59.199 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42915.log.gz 2024-09-16T05:03:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29854.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.29854.log.gz --verbose 2024-09-16T05:03:59.199 INFO:teuthology.orchestra.run.smithi012.stderr: -- /var/log/ceph/ceph-client.admin.31578.log 2024-09-16T05:03:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38592.log 2024-09-16T05:03:59.199 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37863.log.gz 2024-09-16T05:03:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36208.log 2024-09-16T05:03:59.199 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31578.log.gz 2024-09-16T05:03:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38592.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38592.log.gz 2024-09-16T05:03:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34947.log 2024-09-16T05:03:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39034.log 2024-09-16T05:03:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36208.log: /var/log/ceph/ceph-client.admin.34947.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36208.log.gz 2024-09-16T05:03:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34507.log 2024-09-16T05:03:59.200 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34947.log.gz 2024-09-16T05:03:59.200 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39034.log.gz 2024-09-16T05:03:59.200 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42577.log 2024-09-16T05:03:59.200 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.34507.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50025.log 2024-09-16T05:03:59.200 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34507.log.gz 2024-09-16T05:03:59.200 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50364.log 2024-09-16T05:03:59.200 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42577.log.gz 2024-09-16T05:03:59.200 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43019.log 2024-09-16T05:03:59.200 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50025.log.gz 2024-09-16T05:03:59.200 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34511.log 2024-09-16T05:03:59.200 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50364.log.gz 2024-09-16T05:03:59.200 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46512.log 2024-09-16T05:03:59.201 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43019.log.gz 2024-09-16T05:03:59.201 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41559.log 2024-09-16T05:03:59.201 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34511.log.gz 2024-09-16T05:03:59.201 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.46512.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40363.log 2024-09-16T05:03:59.205 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46512.log.gz 2024-09-16T05:03:59.205 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45575.log 2024-09-16T05:03:59.205 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41559.log.gz 2024-09-16T05:03:59.205 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44714.log 2024-09-16T05:03:59.205 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40363.log.gz 2024-09-16T05:03:59.205 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29828.log 2024-09-16T05:03:59.205 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45575.log.gz 2024-09-16T05:03:59.205 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.44714.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49583.log 2024-09-16T05:03:59.205 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44714.log.gz 2024-09-16T05:03:59.205 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37890.log 2024-09-16T05:03:59.206 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29828.log.gz 2024-09-16T05:03:59.206 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49583.log.gz 2024-09-16T05:03:59.206 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44636.log 2024-09-16T05:03:59.206 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38462.log 2024-09-16T05:03:59.206 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37890.log.gz 2024-09-16T05:03:59.206 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47240.log 2024-09-16T05:03:59.206 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44636.log.gz 2024-09-16T05:03:59.206 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46538.log 2024-09-16T05:03:59.206 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38462.log.gz 2024-09-16T05:03:59.206 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43646.log 2024-09-16T05:03:59.206 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47240.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47240.log.gz 2024-09-16T05:03:59.206 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51850.log 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46538.log.gz 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39372.log 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43646.log.gz 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49661.log 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51850.log.gz 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42213.log 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39372.log.gz 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45653.log 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49661.log.gz 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32943.log 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42213.log.gz 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49167.log 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45653.log.gz 2024-09-16T05:03:59.207 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29419.log 2024-09-16T05:03:59.208 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32943.log.gz 2024-09-16T05:03:59.208 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48724.log 2024-09-16T05:03:59.208 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49167.log.gz 2024-09-16T05:03:59.208 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44818.log 2024-09-16T05:03:59.208 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29419.log.gz 2024-09-16T05:03:59.208 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48724.log.gz 2024-09-16T05:03:59.208 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43149.log 2024-09-16T05:03:59.208 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38696.log 2024-09-16T05:03:59.208 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44818.log.gz 2024-09-16T05:03:59.208 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51668.log 2024-09-16T05:03:59.208 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43149.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43149.log.gz 2024-09-16T05:03:59.208 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45861.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38696.log.gz 2024-09-16T05:03:59.208 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-09-16T05:03:59.209 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35025.log 2024-09-16T05:03:59.209 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51668.log.gz 2024-09-16T05:03:59.209 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47292.log 2024-09-16T05:03:59.209 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45861.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45861.log.gz 2024-09-16T05:03:59.209 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45105.log 2024-09-16T05:03:59.209 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35025.log.gz 2024-09-16T05:03:59.209 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38228.log 2024-09-16T05:03:59.209 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47292.log.gz 2024-09-16T05:03:59.209 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34377.log 2024-09-16T05:03:59.209 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45105.log.gz 2024-09-16T05:03:59.209 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35675.log 2024-09-16T05:03:59.209 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38228.log.gz 2024-09-16T05:03:59.209 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39921.log 2024-09-16T05:03:59.210 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34377.log.gz 2024-09-16T05:03:59.210 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45183.log 2024-09-16T05:03:59.210 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35675.log.gz 2024-09-16T05:03:59.210 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51720.log 2024-09-16T05:03:59.210 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39921.log.gz 2024-09-16T05:03:59.210 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34713.log 2024-09-16T05:03:59.210 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45183.log.gz 2024-09-16T05:03:59.210 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42395.log 2024-09-16T05:03:59.210 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51720.log.gz 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42551.log 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34713.log.gz 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42395.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.42395.log.gz --verbose 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr: -- /var/log/ceph/ceph-client.admin.48906.log 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33095.log 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42551.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42551.log.gz 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39973.log 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48906.log.gz 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29550.log 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33095.log.gz 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31622.log 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39973.log.gz 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33121.log 2024-09-16T05:03:59.211 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29550.log.gz 2024-09-16T05:03:59.212 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41481.log 0.0% 2024-09-16T05:03:59.212 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31622.log.gz 2024-09-16T05:03:59.212 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35129.log 2024-09-16T05:03:59.212 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33121.log.gz 2024-09-16T05:03:59.212 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41481.log.gz 2024-09-16T05:03:59.212 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38254.log 2024-09-16T05:03:59.212 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43568.log 2024-09-16T05:03:59.212 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35129.log.gz 2024-09-16T05:03:59.212 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39190.log 2024-09-16T05:03:59.212 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38254.log.gz 2024-09-16T05:03:59.212 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43568.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.43568.log.gz 2024-09-16T05:03:59.212 INFO:teuthology.orchestra.run.smithi012.stderr: /var/log/ceph/ceph-client.admin.50704.log 2024-09-16T05:03:59.212 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31490.log 2024-09-16T05:03:59.213 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39190.log.gz 2024-09-16T05:03:59.213 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.50704.log: /var/log/ceph/ceph-client.admin.34791.log 2024-09-16T05:03:59.213 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50704.log.gz 2024-09-16T05:03:59.213 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31490.log.gz 2024-09-16T05:03:59.213 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43906.log 2024-09-16T05:03:59.213 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34791.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.34791.log.gz 2024-09-16T05:03:59.213 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.30015.log 2024-09-16T05:03:59.213 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41429.log 2024-09-16T05:03:59.213 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43906.log.gz 2024-09-16T05:03:59.213 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29464.log 2024-09-16T05:03:59.213 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30015.log.gz 2024-09-16T05:03:59.213 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50156.log 0.0% 2024-09-16T05:03:59.213 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41429.log.gz 2024-09-16T05:03:59.214 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49557.log 2024-09-16T05:03:59.214 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29464.log.gz 2024-09-16T05:03:59.214 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35856.log 2024-09-16T05:03:59.214 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50156.log.gz 2024-09-16T05:03:59.214 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40909.log 2024-09-16T05:03:59.214 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49557.log.gz 2024-09-16T05:03:59.214 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29394.log 2024-09-16T05:03:59.214 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35856.log.gz 2024-09-16T05:03:59.214 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.40909.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41533.log 2024-09-16T05:03:59.214 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40909.log.gz 2024-09-16T05:03:59.214 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49115.log 2024-09-16T05:03:59.214 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29394.log.gz 2024-09-16T05:03:59.214 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49947.log 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41533.log.gz 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40467.log 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49115.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49115.log.gz 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39503.log 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49947.log.gz 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39661.log 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40467.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40467.log.gz 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41716.log 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39503.log.gz 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51616.log 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39661.log.gz 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.41716.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44923.log 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41716.log.gz 2024-09-16T05:03:59.215 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38904.log 2024-09-16T05:03:59.216 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51616.log.gz 2024-09-16T05:03:59.216 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34765.log 2024-09-16T05:03:59.216 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44923.log.gz 2024-09-16T05:03:59.345 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34999.log 2024-09-16T05:03:59.346 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38904.log.gz 2024-09-16T05:03:59.346 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40389.log 2024-09-16T05:03:59.346 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34765.log.gz 2024-09-16T05:03:59.346 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43516.log 2024-09-16T05:03:59.346 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34999.log.gz 2024-09-16T05:03:59.346 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41039.log 2024-09-16T05:03:59.346 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40389.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40389.log.gz 2024-09-16T05:03:59.346 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46095.log 2024-09-16T05:03:59.346 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43516.log.gz 2024-09-16T05:03:59.346 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37499.log 2024-09-16T05:03:59.346 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41039.log.gz 2024-09-16T05:03:59.346 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35077.log 2024-09-16T05:03:59.347 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46095.log.gz 2024-09-16T05:03:59.347 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47527.log 2024-09-16T05:03:59.347 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37499.log.gz 2024-09-16T05:03:59.347 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31864.log 2024-09-16T05:03:59.347 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35077.log.gz 2024-09-16T05:03:59.347 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36274.log 2024-09-16T05:03:59.347 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47527.log.gz 2024-09-16T05:03:59.347 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36230.log 2024-09-16T05:03:59.347 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31864.log.gz 2024-09-16T05:03:59.347 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36274.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.44192.log -- replaced with /var/log/ceph/ceph-client.admin.36274.log.gz 2024-09-16T05:03:59.347 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-09-16T05:03:59.347 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30685.log 2024-09-16T05:03:59.347 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36230.log.gz 2024-09-16T05:03:59.348 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44192.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44192.log.gz 2024-09-16T05:03:59.348 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51044.log 2024-09-16T05:03:59.348 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43698.log 2024-09-16T05:03:59.348 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30685.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30685.log.gz 2024-09-16T05:03:59.348 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46173.log 2024-09-16T05:03:59.348 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51044.log.gz 2024-09-16T05:03:59.348 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35471.log 2024-09-16T05:03:59.348 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43698.log.gz 2024-09-16T05:03:59.348 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48567.log 2024-09-16T05:03:59.348 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46173.log.gz 2024-09-16T05:03:59.348 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48359.log 2024-09-16T05:03:59.348 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35471.log.gz 2024-09-16T05:03:59.348 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42187.log 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48567.log.gz 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41247.log 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48359.log.gz 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48880.log 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42187.log.gz 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46616.log 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41247.log.gz 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34843.log 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48880.log.gz 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.46616.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43256.log 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46616.log.gz 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35878.log 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34843.log.gz 2024-09-16T05:03:59.349 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38124.log 2024-09-16T05:03:59.350 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43256.log.gz 2024-09-16T05:03:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51174.log 2024-09-16T05:03:59.350 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35878.log.gz 2024-09-16T05:03:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34895.log 2024-09-16T05:03:59.350 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38124.log.gz 2024-09-16T05:03:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29802.log 2024-09-16T05:03:59.350 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51174.log.gz 2024-09-16T05:03:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45913.log 2024-09-16T05:03:59.350 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34895.log.gz 2024-09-16T05:03:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29802.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29802.log.gz 2024-09-16T05:03:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41586.log 2024-09-16T05:03:59.350 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35311.log 2024-09-16T05:03:59.350 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45913.log.gz 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36186.log 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41586.log.gz 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35311.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.35311.log.gz /var/log/ceph/ceph-client.admin.42759.log 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42421.log 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36186.log.gz 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35966.log 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42759.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42759.log.gz 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38332.log 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42421.log.gz 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45209.log 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35966.log.gz 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34817.log 2024-09-16T05:03:59.351 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38332.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38332.log.gz 2024-09-16T05:03:59.352 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45209.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.45209.log.gz 2024-09-16T05:03:59.352 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42054.log 2024-09-16T05:03:59.352 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45523.log 2024-09-16T05:03:59.352 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34817.log.gz 2024-09-16T05:03:59.352 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39999.log 2024-09-16T05:03:59.352 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42054.log.gz 2024-09-16T05:03:59.352 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45523.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45523.log.gz 2024-09-16T05:03:59.352 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49245.log 2024-09-16T05:03:59.352 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35207.log 2024-09-16T05:03:59.352 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39999.log.gz 2024-09-16T05:03:59.352 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32835.log 2024-09-16T05:03:59.352 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49245.log.gz 2024-09-16T05:03:59.352 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45731.log 2024-09-16T05:03:59.353 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35207.log.gz 2024-09-16T05:03:59.353 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41169.log 2024-09-16T05:03:59.353 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32835.log.gz 2024-09-16T05:03:59.353 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30400.log 2024-09-16T05:03:59.353 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45731.log.gz 2024-09-16T05:03:59.353 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44036.log 2024-09-16T05:03:59.353 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41169.log.gz 2024-09-16T05:03:59.353 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39425.log 2024-09-16T05:03:59.353 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30400.log.gz 2024-09-16T05:03:59.353 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30351.log 2024-09-16T05:03:59.353 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44036.log.gz 2024-09-16T05:03:59.353 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39425.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39425.log.gz 2024-09-16T05:03:59.353 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48151.log 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40181.log 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30351.log.gz 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35585.log 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48151.log.gz 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40181.log.gz 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45393.log 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31468.log 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35585.log.gz 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45965.log 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45393.log.gz 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35571.log 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31468.log.gz 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44140.log 2024-09-16T05:03:59.354 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45965.log.gz 2024-09-16T05:03:59.355 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30778.log 2024-09-16T05:03:59.355 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35571.log.gz 2024-09-16T05:03:59.355 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48229.log 2024-09-16T05:03:59.355 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44140.log.gz 2024-09-16T05:03:59.355 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37447.log 2024-09-16T05:03:59.355 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30778.log.gz 2024-09-16T05:03:59.355 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39294.log 2024-09-16T05:03:59.355 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48229.log.gz 2024-09-16T05:03:59.355 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51252.log 2024-09-16T05:03:59.355 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37447.log.gz 2024-09-16T05:03:59.355 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39294.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-16T05:03:59.355 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39294.log.gz 2024-09-16T05:03:59.355 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39451.log 2024-09-16T05:03:59.356 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51252.log.gz 2024-09-16T05:03:59.356 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37473.log 2024-09-16T05:03:59.356 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43360.log 2024-09-16T05:03:59.356 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39451.log.gz 2024-09-16T05:03:59.356 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47110.log 2024-09-16T05:03:59.356 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37473.log.gz 2024-09-16T05:03:59.356 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41221.log 2024-09-16T05:03:59.356 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43360.log.gz 2024-09-16T05:03:59.356 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40597.log 2024-09-16T05:03:59.356 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47110.log.gz 2024-09-16T05:03:59.356 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33147.log 2024-09-16T05:03:59.356 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41221.log.gz 2024-09-16T05:03:59.356 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31108.log 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40597.log.gz 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33147.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33147.log.gz 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34559.log 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49869.log 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31108.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31108.log.gz 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35790.log 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34559.log.gz 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46798.log 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49869.log.gz 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.35790.log: /var/log/ceph/ceph-client.admin.41351.log 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35790.log.gz 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47032.log 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46798.log.gz 2024-09-16T05:03:59.357 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41351.log.gz 2024-09-16T05:03:59.358 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37395.log 2024-09-16T05:03:59.358 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52084.log 2024-09-16T05:03:59.358 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47032.log.gz 2024-09-16T05:03:59.358 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46850.log 2024-09-16T05:03:59.358 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37395.log.gz 2024-09-16T05:03:59.358 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51148.log 2024-09-16T05:03:59.358 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52084.log.gz 2024-09-16T05:03:59.358 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39060.log 2024-09-16T05:03:59.358 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46850.log.gz 2024-09-16T05:03:59.358 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34251.log 2024-09-16T05:03:59.358 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51148.log.gz 2024-09-16T05:03:59.358 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48073.log 2024-09-16T05:03:59.358 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39060.log.gz 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43490.log 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34251.log.gz 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42080.log 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48073.log.gz 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43490.log: -- /var/log/ceph/ceph-client.admin.45079.log 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43490.log.gz 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37785.log 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42080.log.gz 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45079.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45079.log.gz 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40311.log 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51304.log 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37785.log.gz 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38384.log 2024-09-16T05:03:59.359 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40311.log.gz 2024-09-16T05:03:59.360 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39398.log 2024-09-16T05:03:59.360 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51304.log.gz 2024-09-16T05:03:59.360 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40961.log 2024-09-16T05:03:59.360 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38384.log.gz 2024-09-16T05:03:59.360 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49219.log 2024-09-16T05:03:59.360 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39398.log.gz 2024-09-16T05:03:59.360 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29627.log 2024-09-16T05:03:59.360 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40961.log.gz 2024-09-16T05:03:59.360 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47683.log 2024-09-16T05:03:59.360 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49219.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49219.log.gz 2024-09-16T05:03:59.360 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49505.log 2024-09-16T05:03:59.360 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29627.log.gz 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47917.log 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47683.log.gz 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50599.log 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49505.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49505.log.gz 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51486.log 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47917.log.gz 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.50599.log: /var/log/ceph/ceph-client.admin.31666.log 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50599.log.gz 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49999.log 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51486.log.gz 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31666.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31666.log.gz 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31886.log 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40545.log 2024-09-16T05:03:59.361 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49999.log.gz 2024-09-16T05:03:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45497.log 2024-09-16T05:03:59.362 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31886.log.gz 2024-09-16T05:03:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47266.log 2024-09-16T05:03:59.362 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40545.log.gz 2024-09-16T05:03:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35469.log 2024-09-16T05:03:59.362 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45497.log.gz 2024-09-16T05:03:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40025.log 2024-09-16T05:03:59.362 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47266.log.gz 2024-09-16T05:03:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37265.log 2024-09-16T05:03:59.362 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35469.log.gz 2024-09-16T05:03:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49973.log 2024-09-16T05:03:59.362 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40025.log.gz 2024-09-16T05:03:59.362 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30711.log 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37265.log.gz 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37837.log 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49973.log.gz 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36098.log 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30711.log.gz 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44324.log 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37837.log.gz 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36098.log.gz 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43438.log 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.44324.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49063.log 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44324.log.gz 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51278.log 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43438.log.gz 2024-09-16T05:03:59.363 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49063.log.gz 2024-09-16T05:03:59.364 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47162.log 2024-09-16T05:03:59.364 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41768.log 2024-09-16T05:03:59.364 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51278.log.gz 2024-09-16T05:03:59.364 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47162.log.gz 2024-09-16T05:03:59.364 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46303.log 2024-09-16T05:03:59.364 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34661.log 2024-09-16T05:03:59.364 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41768.log.gz 2024-09-16T05:03:59.364 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31446.log 2024-09-16T05:03:59.364 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46303.log.gz 2024-09-16T05:03:59.364 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34661.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.42447.log 2024-09-16T05:03:59.364 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34661.log.gz 2024-09-16T05:03:59.364 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31512.log 2024-09-16T05:03:59.364 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31446.log.gz 2024-09-16T05:03:59.365 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46407.log 2024-09-16T05:03:59.365 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42447.log.gz 2024-09-16T05:03:59.365 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31512.log.gz 2024-09-16T05:03:59.365 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46720.log 2024-09-16T05:03:59.365 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47188.log 2024-09-16T05:03:59.365 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46407.log.gz 2024-09-16T05:03:59.365 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46720.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.46720.log.gz 2024-09-16T05:03:59.365 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50338.log 2024-09-16T05:03:59.365 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47188.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-16T05:03:59.365 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47188.log.gz 2024-09-16T05:03:59.365 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50338.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.50338.log.gz 2024-09-16T05:03:59.365 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51928.log 2024-09-16T05:03:59.365 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42967.log 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51928.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51928.log.gz 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38982.log 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.42967.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46147.log 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42967.log.gz 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40441.log 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38982.log.gz 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46147.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46147.log.gz 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43542.log 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40441.log.gz 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50730.log 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46277.log 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43542.log.gz 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38046.log 2024-09-16T05:03:59.366 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50730.log.gz 2024-09-16T05:03:59.367 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44428.log 2024-09-16T05:03:59.367 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46277.log.gz 2024-09-16T05:03:59.367 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38046.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38046.log.gz 2024-09-16T05:03:59.367 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46954.log 2024-09-16T05:03:59.367 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44428.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.44428.log.gz /var/log/ceph/ceph-client.admin.35922.log 2024-09-16T05:03:59.367 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-09-16T05:03:59.367 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46954.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T05:03:59.367 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46954.log.gz 2024-09-16T05:03:59.367 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47136.log 2024-09-16T05:03:59.367 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35922.log.gz 2024-09-16T05:03:59.367 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30529.log 2024-09-16T05:03:59.367 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47136.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47136.log.gz 2024-09-16T05:03:59.367 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50964.log 2024-09-16T05:03:59.368 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30844.log 2024-09-16T05:03:59.368 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30529.log.gz 2024-09-16T05:03:59.368 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47813.log 2024-09-16T05:03:59.368 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50964.log.gz 2024-09-16T05:03:59.368 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30844.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30844.log.gz 2024-09-16T05:03:59.368 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31064.log 2024-09-16T05:03:59.368 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50442.log 2024-09-16T05:03:59.368 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47813.log.gz 2024-09-16T05:03:59.368 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31064.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31064.log.gz -5 2024-09-16T05:03:59.368 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48802.log 2024-09-16T05:03:59.368 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43880.log 2024-09-16T05:03:59.368 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50442.log.gz 2024-09-16T05:03:59.368 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51772.log 2024-09-16T05:03:59.369 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48802.log.gz 2024-09-16T05:03:59.369 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31130.log 2024-09-16T05:03:59.369 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43880.log.gz 2024-09-16T05:03:59.369 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51772.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51772.log.gz 2024-09-16T05:03:59.369 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47735.log 2024-09-16T05:03:59.369 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38826.log 2024-09-16T05:03:59.369 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31130.log.gz 2024-09-16T05:03:59.369 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47735.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47735.log.gz 2024-09-16T05:03:59.369 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51746.log 2024-09-16T05:03:59.369 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39242.log 2024-09-16T05:03:59.369 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38826.log.gz 2024-09-16T05:03:59.369 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42811.log 2024-09-16T05:03:59.369 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51746.log.gz 2024-09-16T05:03:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43828.log 2024-09-16T05:03:59.370 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39242.log.gz 2024-09-16T05:03:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49713.log 2024-09-16T05:03:59.370 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42811.log.gz 2024-09-16T05:03:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48385.log 2024-09-16T05:03:59.370 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43828.log.gz 2024-09-16T05:03:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31306.log 2024-09-16T05:03:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49713.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49713.log.gz 2024-09-16T05:03:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30954.log 2024-09-16T05:03:59.370 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48385.log.gz 2024-09-16T05:03:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31306.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.31306.log.gz 2024-09-16T05:03:59.370 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.29698.log 2024-09-16T05:03:59.370 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30954.log.gz 2024-09-16T05:03:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30010.log 2024-09-16T05:03:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30325.log 2024-09-16T05:03:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29698.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29698.log.gz 2024-09-16T05:03:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30010.log.gz 2024-09-16T05:03:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42265.log 2024-09-16T05:03:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48437.log 2024-09-16T05:03:59.371 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30325.log.gz 2024-09-16T05:03:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42265.log.gz 2024-09-16T05:03:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46564.log 2024-09-16T05:03:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37811.log 2024-09-16T05:03:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48437.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48437.log.gz 2024-09-16T05:03:59.371 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52111.log 2024-09-16T05:03:59.371 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46564.log.gz 2024-09-16T05:03:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34198.log 2024-09-16T05:03:59.372 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37811.log.gz 2024-09-16T05:03:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45887.log 2024-09-16T05:03:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52111.log.gz 2024-09-16T05:03:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25447.log 2024-09-16T05:03:59.372 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34198.log.gz 2024-09-16T05:03:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48776.log 2024-09-16T05:03:59.372 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45887.log.gz 2024-09-16T05:03:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.25447.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.37343.log -- replaced with /var/log/ceph/ceph-client.admin.25447.log.gz 2024-09-16T05:03:59.372 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-09-16T05:03:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40077.log 2024-09-16T05:03:59.372 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48776.log.gz 2024-09-16T05:03:59.372 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25375.log 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37343.log.gz 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40077.log.gz 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44402.log 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.25375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38020.log 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25375.log.gz 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42239.log 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44402.log.gz 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38020.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38020.log.gz 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37629.log 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36164.log 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42239.log.gz 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42369.log 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37629.log.gz 2024-09-16T05:03:59.373 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45809.log 2024-09-16T05:03:59.374 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36164.log.gz 2024-09-16T05:03:59.374 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41117.log 2024-09-16T05:03:59.374 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42369.log.gz 2024-09-16T05:03:59.374 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5/var/log/ceph/ceph-client.admin.45809.log: --verbose -- /var/log/ceph/ceph-client.admin.48932.log 2024-09-16T05:03:59.374 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45809.log.gz 2024-09-16T05:03:59.374 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49843.log 2024-09-16T05:03:59.374 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41117.log.gz 2024-09-16T05:03:59.374 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44662.log 2024-09-16T05:03:59.374 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48932.log.gz 2024-09-16T05:03:59.374 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34199.log 2024-09-16T05:03:59.374 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49843.log.gz 2024-09-16T05:03:59.374 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34458.log 2024-09-16T05:03:59.374 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44662.log.gz 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34199.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34199.log.gz 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47553.log 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34458.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34458.log.gz 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31556.log 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31534.log 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47553.log.gz 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51018.log 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31556.log.gz 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43854.log 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31534.log.gz 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38930.log 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51018.log.gz 2024-09-16T05:03:59.375 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47865.log 2024-09-16T05:03:59.376 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43854.log.gz 2024-09-16T05:03:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49453.log 2024-09-16T05:03:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38930.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38930.log.gz 2024-09-16T05:03:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47865.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47865.log.gz 2024-09-16T05:03:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45757.log 2024-09-16T05:03:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25340.log 2024-09-16T05:03:59.376 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49453.log.gz 2024-09-16T05:03:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45757.log.gz 2024-09-16T05:03:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45705.log 2024-09-16T05:03:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.25340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42132.log 2024-09-16T05:03:59.376 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25340.log.gz 2024-09-16T05:03:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43594.log 2024-09-16T05:03:59.376 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45705.log.gz 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39164.log 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42132.log.gz 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48646.log 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43594.log.gz 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38514.log 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39164.log.gz 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45549.log 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48646.log.gz 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35746.log 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38514.log.gz 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42889.log 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45549.log.gz 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34325.log 2024-09-16T05:03:59.377 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35746.log.gz 2024-09-16T05:03:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39086.log 2024-09-16T05:03:59.378 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42889.log.gz 2024-09-16T05:03:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49193.log 2024-09-16T05:03:59.378 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34325.log.gz 2024-09-16T05:03:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35900.log 2024-09-16T05:03:59.378 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39086.log.gz 2024-09-16T05:03:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50494.log 2024-09-16T05:03:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49193.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49193.log.gz 2024-09-16T05:03:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43308.log 2024-09-16T05:03:59.378 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35900.log.gz 2024-09-16T05:03:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35834.log 2024-09-16T05:03:59.378 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50494.log.gz 2024-09-16T05:03:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51382.log 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43308.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43308.log.gz 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38878.log 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35834.log.gz 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34147.log 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51382.log.gz 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31086.log 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38878.log.gz 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35155.log 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34147.log.gz 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr: 95.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30800.log 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31086.log.gz 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32891.log 2024-09-16T05:03:59.379 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35155.log.gz 2024-09-16T05:03:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.30800.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31152.log 2024-09-16T05:03:59.380 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30800.log.gz 2024-09-16T05:03:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47969.log 2024-09-16T05:03:59.380 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32891.log.gz 2024-09-16T05:03:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31908.log 2024-09-16T05:03:59.380 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31152.log.gz 2024-09-16T05:03:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49323.log 2024-09-16T05:03:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47969.log.gz 2024-09-16T05:03:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31240.log 2024-09-16T05:03:59.380 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31908.log.gz 2024-09-16T05:03:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50547.log 2024-09-16T05:03:59.380 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49323.log.gz 2024-09-16T05:03:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42655.log 2024-09-16T05:03:59.381 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31240.log.gz 2024-09-16T05:03:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48541.log 2024-09-16T05:03:59.381 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50547.log.gz 2024-09-16T05:03:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39112.log 2024-09-16T05:03:59.381 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42655.log.gz 2024-09-16T05:03:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41091.log 2024-09-16T05:03:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48541.log.gz 2024-09-16T05:03:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30633.log 2024-09-16T05:03:59.381 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39112.log.gz 2024-09-16T05:03:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41091.log.gz 2024-09-16T05:03:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49297.log 2024-09-16T05:03:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40415.log 2024-09-16T05:03:59.381 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30633.log.gz 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35363.log 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49297.log.gz 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42343.log 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40415.log.gz 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47422.log 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35363.log.gz 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46902.log 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42343.log.gz 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47422.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47422.log.gz 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48125.log 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46902.log.gz 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40285.log 2024-09-16T05:03:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40883.log 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48125.log.gz 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50625.log 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40285.log.gz 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41794.log 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40883.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40883.log.gz 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50286.log 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50625.log.gz 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48489.log 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41794.log.gz 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50286.log: -- /var/log/ceph/ceph-client.admin.35259.log 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50286.log.gz 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38540.log 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48489.log.gz 2024-09-16T05:03:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51954.log 2024-09-16T05:03:59.384 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35259.log.gz 2024-09-16T05:03:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46199.log 2024-09-16T05:03:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38540.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38540.log.gz 2024-09-16T05:03:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52032.log 2024-09-16T05:03:59.384 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51954.log.gz 2024-09-16T05:03:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41325.log 2024-09-16T05:03:59.384 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46199.log.gz 2024-09-16T05:03:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43230.log 2024-09-16T05:03:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52032.log.gz 2024-09-16T05:03:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40493.log 2024-09-16T05:03:59.384 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41325.log.gz 2024-09-16T05:03:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51590.log 2024-09-16T05:03:59.384 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43230.log.gz 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45053.log 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40493.log.gz 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42733.log 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51590.log.gz 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45053.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.30173.log -- replaced with /var/log/ceph/ceph-client.admin.45053.log.gz 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45991.log 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42733.log.gz 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30173.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.38748.log -- replaced with /var/log/ceph/ceph-client.admin.30173.log.gz 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41377.log 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45991.log.gz 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29628.log 2024-09-16T05:03:59.385 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38748.log.gz 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48828.log 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41377.log.gz 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.29628.log: /var/log/ceph/ceph-client.admin.42785.log 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29628.log.gz 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41455.log 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48828.log.gz 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42002.log 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42785.log.gz 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35768.log 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41455.log.gz 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39320.log 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42002.log.gz 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50573.log 2024-09-16T05:03:59.386 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35768.log.gz 2024-09-16T05:03:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39947.log 2024-09-16T05:03:59.387 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39320.log.gz 2024-09-16T05:03:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33173.log 2024-09-16T05:03:59.387 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50573.log.gz 2024-09-16T05:03:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30429.log 2024-09-16T05:03:59.387 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39947.log.gz 2024-09-16T05:03:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.33173.log: /var/log/ceph/ceph-client.admin.37759.log 2024-09-16T05:03:59.387 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33173.log.gz 2024-09-16T05:03:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38306.log 2024-09-16T05:03:59.387 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30429.log.gz 2024-09-16T05:03:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37759.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21430.log 2024-09-16T05:03:59.387 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37759.log.gz 2024-09-16T05:03:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47839.log 2024-09-16T05:03:59.388 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38306.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38306.log.gz 2024-09-16T05:03:59.388 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.tmp-client.admin.21430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47214.log 2024-09-16T05:03:59.388 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21430.log.gz 2024-09-16T05:03:59.388 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39869.log 2024-09-16T05:03:59.388 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47839.log.gz 2024-09-16T05:03:59.388 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47995.log 2024-09-16T05:03:59.388 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47214.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47214.log.gz 2024-09-16T05:03:59.388 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30976.log 2024-09-16T05:03:59.388 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39869.log.gz 2024-09-16T05:03:59.388 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37994.log 2024-09-16T05:03:59.388 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47995.log.gz 2024-09-16T05:03:59.388 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31174.log 2024-09-16T05:03:59.388 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30976.log.gz 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45287.log 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37994.log.gz 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44610.log 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31174.log.gz 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44088.log 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45287.log.gz 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36142.log 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44610.log.gz 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44088.log.gz 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43175.log 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29940.log 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36142.log.gz 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38852.log 2024-09-16T05:03:59.389 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43175.log.gz 2024-09-16T05:03:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29936.log 2024-09-16T05:03:59.390 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29940.log.gz 2024-09-16T05:03:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38956.log 2024-09-16T05:03:59.390 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38852.log.gz 2024-09-16T05:03:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40519.log 2024-09-16T05:03:59.390 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29936.log.gz 2024-09-16T05:03:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29937.log 2024-09-16T05:03:59.390 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38956.log.gz 2024-09-16T05:03:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31331.log 2024-09-16T05:03:59.390 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40519.log.gz 2024-09-16T05:03:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29937.log.gz 2024-09-16T05:03:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30910.log 2024-09-16T05:03:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29906.log 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31331.log.gz 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31020.log 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30910.log.gz 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43045.log 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29906.log.gz 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41872.log 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31020.log.gz 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29725.log 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43045.log.gz 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34635.log 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41872.log.gz 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30270.log 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29725.log.gz 2024-09-16T05:03:59.391 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40857.log 2024-09-16T05:03:59.392 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34635.log.gz 2024-09-16T05:03:59.392 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44975.log 2024-09-16T05:03:59.392 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30270.log.gz 2024-09-16T05:03:59.392 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35724.log 2024-09-16T05:03:59.392 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40857.log.gz 2024-09-16T05:03:59.392 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34117.log 2024-09-16T05:03:59.392 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44975.log.gz 2024-09-16T05:03:59.392 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32939.log 2024-09-16T05:03:59.392 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35724.log.gz 2024-09-16T05:03:59.392 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29549.log 2024-09-16T05:03:59.392 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34117.log.gz 2024-09-16T05:03:59.392 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31218.log 2024-09-16T05:03:59.392 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32939.log.gz 2024-09-16T05:03:59.393 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43097.log 2024-09-16T05:03:59.393 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29549.log.gz 2024-09-16T05:03:59.393 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49739.log 2024-09-16T05:03:59.393 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31218.log.gz 2024-09-16T05:03:59.393 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42291.log 2024-09-16T05:03:59.393 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43097.log.gz 2024-09-16T05:03:59.393 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49739.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49739.log.gz 2024-09-16T05:03:59.393 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43412.log 2024-09-16T05:03:59.393 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51902.log 2024-09-16T05:03:59.393 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42291.log.gz 2024-09-16T05:03:59.393 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44350.log 2024-09-16T05:03:59.393 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43412.log.gz 2024-09-16T05:03:59.393 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51902.log.gz 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48750.log 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51564.log 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44350.log.gz 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46355.log 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48750.log.gz 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49349.log 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51564.log.gz 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41820.log 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46355.log.gz 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50104.log 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49349.log.gz 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45157.log 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41820.log.gz 2024-09-16T05:03:59.394 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44897.log 2024-09-16T05:03:59.395 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50104.log.gz 2024-09-16T05:03:59.395 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.45157.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42028.log 2024-09-16T05:03:59.395 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45157.log.gz 2024-09-16T05:03:59.395 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45679.log 2024-09-16T05:03:59.395 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44897.log.gz 2024-09-16T05:03:59.395 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48515.log 2024-09-16T05:03:59.395 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42028.log.gz 2024-09-16T05:03:59.395 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42525.log 2024-09-16T05:03:59.395 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45679.log.gz 2024-09-16T05:03:59.395 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46069.log 2024-09-16T05:03:59.395 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48515.log.gz 2024-09-16T05:03:59.395 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48620.log 2024-09-16T05:03:59.396 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42525.log.gz 2024-09-16T05:03:59.396 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37707.log 2024-09-16T05:03:59.396 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46069.log.gz 2024-09-16T05:03:59.396 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30174.log 2024-09-16T05:03:59.396 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48620.log.gz 2024-09-16T05:03:59.396 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42603.log 2024-09-16T05:03:59.396 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37707.log.gz 2024-09-16T05:03:59.396 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5/var/log/ceph/ceph-client.admin.30174.log: --verbose -- /var/log/ceph/ceph-client.admin.39477.log 2024-09-16T05:03:59.396 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30174.log.gz 2024-09-16T05:03:59.396 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31776.log 2024-09-16T05:03:59.397 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42603.log.gz 2024-09-16T05:03:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39477.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39477.log.gz 2024-09-16T05:03:59.397 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41638.log 2024-09-16T05:03:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.31776.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50182.log 2024-09-16T05:03:59.397 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31776.log.gz 2024-09-16T05:03:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47006.log 2024-09-16T05:03:59.397 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41638.log.gz 2024-09-16T05:03:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50182.log.gz 2024-09-16T05:03:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45261.log 2024-09-16T05:03:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46824.log 2024-09-16T05:03:59.397 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47006.log.gz 2024-09-16T05:03:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51512.log 2024-09-16T05:03:59.397 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45261.log.gz 2024-09-16T05:03:59.398 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46824.log.gz 2024-09-16T05:03:59.398 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-16T05:03:59.398 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49427.log 2024-09-16T05:03:59.398 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51512.log.gz 2024-09-16T05:03:59.398 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36054.log 2024-09-16T05:03:59.398 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40805.log 2024-09-16T05:03:59.398 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49427.log.gz 2024-09-16T05:03:59.398 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.36054.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44480.log 2024-09-16T05:03:59.398 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36054.log.gz 2024-09-16T05:03:59.398 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40051.log 2024-09-16T05:03:59.398 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40805.log.gz 2024-09-16T05:03:59.398 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31644.log 2024-09-16T05:03:59.398 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44480.log.gz 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34351.log 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40051.log.gz 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46329.log 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31644.log.gz 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36010.log 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34351.log.gz 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43123.log 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46329.log.gz 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30427.log 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36010.log.gz 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48021.log 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43123.log.gz 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31357.log 2024-09-16T05:03:59.399 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30427.log.gz 2024-09-16T05:03:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45027.log 2024-09-16T05:03:59.400 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48021.log.gz 2024-09-16T05:03:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38618.log 2024-09-16T05:03:59.400 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31357.log.gz 2024-09-16T05:03:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43724.log 2024-09-16T05:03:59.400 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45027.log.gz 2024-09-16T05:03:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50756.log 2024-09-16T05:03:59.400 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38618.log.gz 2024-09-16T05:03:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39138.log 2024-09-16T05:03:59.400 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43724.log.gz 2024-09-16T05:03:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39607.log/var/log/ceph/ceph-client.admin.50756.log: 2024-09-16T05:03:59.400 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50756.log.gz 2024-09-16T05:03:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25292.log 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39138.log.gz 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37916.log 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39607.log.gz 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.25292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43984.log 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25292.log.gz 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45419.log 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37916.log.gz 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43984.log.gz 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43334.log 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50651.log 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45419.log.gz 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43334.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43334.log.gz -5 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43750.log 2024-09-16T05:03:59.401 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48854.log 2024-09-16T05:03:59.402 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50651.log.gz 2024-09-16T05:03:59.402 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52058.log 2024-09-16T05:03:59.402 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43750.log.gz 2024-09-16T05:03:59.402 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45445.log 2024-09-16T05:03:59.402 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48854.log.gz 2024-09-16T05:03:59.402 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31402.log 2024-09-16T05:03:59.402 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52058.log.gz 2024-09-16T05:03:59.402 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39843.log 2024-09-16T05:03:59.402 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45445.log.gz 2024-09-16T05:03:59.402 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52135.log 2024-09-16T05:03:59.402 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31402.log.gz 2024-09-16T05:03:59.402 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40207.log 2024-09-16T05:03:59.402 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39843.log.gz 2024-09-16T05:03:59.403 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37551.log 2024-09-16T05:03:59.403 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52135.log.gz 2024-09-16T05:03:59.403 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41898.log 2024-09-16T05:03:59.403 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40207.log.gz 2024-09-16T05:03:59.403 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39713.log 2024-09-16T05:03:59.403 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37551.log.gz 2024-09-16T05:03:59.403 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41898.log.gz 2024-09-16T05:03:59.403 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51876.log 2024-09-16T05:03:59.403 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39713.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.39713.log.gz --verbose 2024-09-16T05:03:59.403 INFO:teuthology.orchestra.run.smithi012.stderr: -- /var/log/ceph/ceph-client.admin.44584.log 2024-09-16T05:03:59.403 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47605.log 2024-09-16T05:03:59.403 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51876.log.gz 2024-09-16T05:03:59.403 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31842.log 2024-09-16T05:03:59.404 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44584.log.gz 2024-09-16T05:03:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51356.log 2024-09-16T05:03:59.404 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47605.log.gz 2024-09-16T05:03:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51070.log 2024-09-16T05:03:59.404 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31842.log.gz 2024-09-16T05:03:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46043.log 2024-09-16T05:03:59.404 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51356.log.gz 2024-09-16T05:03:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48281.log 2024-09-16T05:03:59.404 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51070.log.gz 2024-09-16T05:03:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31930.log 2024-09-16T05:03:59.404 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46043.log.gz 2024-09-16T05:03:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47370.log 2024-09-16T05:03:59.404 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48281.log.gz 2024-09-16T05:03:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49817.log 2024-09-16T05:03:59.405 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31930.log.gz 2024-09-16T05:03:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.47370.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30866.log 2024-09-16T05:03:59.405 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47370.log.gz 2024-09-16T05:03:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50051.log 2024-09-16T05:03:59.405 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49817.log.gz 2024-09-16T05:03:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50886.log 2024-09-16T05:03:59.405 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30866.log.gz 2024-09-16T05:03:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34299.log 2024-09-16T05:03:59.405 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50051.log.gz 2024-09-16T05:03:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40987.log 2024-09-16T05:03:59.405 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50886.log.gz 2024-09-16T05:03:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40155.log 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34299.log.gz 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40987.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40987.log.gz 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49401.log 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31952.log 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40155.log.gz 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37239.log 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49401.log.gz 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48047.log 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31952.log.gz 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37733.log 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37239.log.gz 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38488.log 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48047.log.gz 2024-09-16T05:03:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31262.log 2024-09-16T05:03:59.407 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37733.log.gz 2024-09-16T05:03:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38566.log 2024-09-16T05:03:59.407 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38488.log.gz 2024-09-16T05:03:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48099.log 2024-09-16T05:03:59.407 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31262.log.gz 2024-09-16T05:03:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30533.log 2024-09-16T05:03:59.407 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38566.log.gz 2024-09-16T05:03:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44218.log 2024-09-16T05:03:59.407 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48099.log.gz 2024-09-16T05:03:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30533.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.30533.log.gz -5 2024-09-16T05:03:59.407 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49765.log 2024-09-16T05:03:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45939.log 2024-09-16T05:03:59.407 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44218.log.gz 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34739.log 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49765.log.gz 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51096.log 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45939.log.gz 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42161.log 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34739.log.gz 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40623.log 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51096.log.gz 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29444.log 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42161.log.gz 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43386.log 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40623.log.gz 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47318.log 2024-09-16T05:03:59.408 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29444.log.gz 2024-09-16T05:03:59.409 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39817.log 2024-09-16T05:03:59.409 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43386.log.gz 2024-09-16T05:03:59.409 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47318.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47318.log.gz 2024-09-16T05:03:59.413 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41612.log 2024-09-16T05:03:59.413 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29620.log 2024-09-16T05:03:59.413 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39817.log.gz 2024-09-16T05:03:59.413 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34921.log 2024-09-16T05:03:59.413 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41612.log.gz 2024-09-16T05:03:59.413 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44949.log 2024-09-16T05:03:59.413 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29620.log.gz 2024-09-16T05:03:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48593.log 2024-09-16T05:03:59.414 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34921.log.gz 2024-09-16T05:03:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42681.log 2024-09-16T05:03:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44949.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44949.log.gz 2024-09-16T05:03:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38670.log 2024-09-16T05:03:59.414 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48593.log.gz 2024-09-16T05:03:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44766.log 2024-09-16T05:03:59.414 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42681.log.gz 2024-09-16T05:03:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43620.log 2024-09-16T05:03:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38670.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38670.log.gz 2024-09-16T05:03:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29880.log 2024-09-16T05:03:59.414 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44766.log.gz 2024-09-16T05:03:59.414 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41664.log 2024-09-16T05:03:59.415 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43620.log.gz 2024-09-16T05:03:59.415 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48958.log 2024-09-16T05:03:59.415 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29880.log.gz 2024-09-16T05:03:59.415 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39555.log 2024-09-16T05:03:59.415 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41664.log.gz 2024-09-16T05:03:59.415 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30096.log 2024-09-16T05:03:59.415 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48958.log.gz 2024-09-16T05:03:59.415 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.39555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31196.log 2024-09-16T05:03:59.415 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39555.log.gz 2024-09-16T05:03:59.415 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51798.log 2024-09-16T05:03:59.415 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30096.log.gz 2024-09-16T05:03:59.415 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40701.log 2024-09-16T05:03:59.415 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31196.log.gz 2024-09-16T05:03:59.416 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44844.log 2024-09-16T05:03:59.416 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51798.log.gz 2024-09-16T05:03:59.416 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.40701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46694.log 2024-09-16T05:03:59.416 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40701.log.gz 2024-09-16T05:03:59.416 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34403.log 2024-09-16T05:03:59.416 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44844.log.gz 2024-09-16T05:03:59.416 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51122.log 2024-09-16T05:03:59.416 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46694.log.gz 2024-09-16T05:03:59.416 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50834.log 2024-09-16T05:03:59.416 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34403.log.gz 2024-09-16T05:03:59.416 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42707.log 2024-09-16T05:03:59.416 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51122.log.gz 2024-09-16T05:03:59.416 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43932.log 2024-09-16T05:03:59.417 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50834.log.gz 2024-09-16T05:03:59.417 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34144.log 2024-09-16T05:03:59.417 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42707.log.gz 2024-09-16T05:03:59.417 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43932.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43932.log.gz 2024-09-16T05:03:59.417 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50782.log 2024-09-16T05:03:59.417 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38150.log 2024-09-16T05:03:59.417 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34144.log.gz 2024-09-16T05:03:59.417 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30756.log 2024-09-16T05:03:59.417 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50782.log.gz 2024-09-16T05:03:59.417 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31688.log 2024-09-16T05:03:59.417 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38150.log.gz 2024-09-16T05:03:59.417 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48672.log 2024-09-16T05:03:59.417 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30756.log.gz 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31424.log 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31688.log.gz 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32861.log 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48672.log.gz 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31424.log.gz 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32991.log 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44244.log 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32861.log.gz 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48463.log 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32991.log.gz 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51330.log 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44244.log.gz 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.48463.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49531.log 2024-09-16T05:03:59.418 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48463.log.gz 2024-09-16T05:03:59.419 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51330.log.gz 2024-09-16T05:03:59.419 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49531.log.gz 2024-09-16T05:03:59.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-16T05:04:00.127 INFO:teuthology.orchestra.run.smithi012.stderr: 95.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T05:04:00.163 INFO:teuthology.orchestra.run.smithi012.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-16T05:04:01.164 INFO:teuthology.orchestra.run.smithi012.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-16T05:04:01.530 INFO:teuthology.orchestra.run.smithi012.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-16T05:04:01.698 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-16T05:06:24.199 INFO:teuthology.orchestra.run.smithi012.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-16T05:08:35.238 INFO:teuthology.orchestra.run.smithi012.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-16T05:08:36.784 INFO:teuthology.orchestra.run.smithi012.stderr: 91.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-16T05:08:36.785 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-09-16T05:08:36.786 INFO:teuthology.orchestra.run.smithi012.stderr:real 4m37.758s 2024-09-16T05:08:36.786 INFO:teuthology.orchestra.run.smithi012.stderr:user 7m19.184s 2024-09-16T05:08:36.786 INFO:teuthology.orchestra.run.smithi012.stderr:sys 0m16.648s 2024-09-16T05:08:36.787 INFO:tasks.ceph:Archiving logs... 2024-09-16T05:08:36.788 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:00:02-rados-squid-distro-default-smithi/7906051/remote/smithi012/log 2024-09-16T05:08:36.789 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T05:09:25.073 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T05:09:25.086 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T05:09:25.087 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T05:09:25.129 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T05:09:25.137 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T05:09:25.138 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T05:09:25.185 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T05:09:25.185 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2024-09-16T05:09:25.185 INFO:teuthology.orchestra.run.smithi012.stdout:*hv01.front.sepi 67.205.162.81 3 u 64 64 377 0.077 +0.340 0.358 2024-09-16T05:09:25.185 INFO:teuthology.orchestra.run.smithi012.stdout:+hv02.front.sepi 63.231.80.2 3 u 7 64 377 0.100 -0.455 0.341 2024-09-16T05:09:25.185 INFO:teuthology.orchestra.run.smithi012.stdout:+hv03.front.sepi 74.6.168.72 3 u 67 64 377 0.089 +2.526 0.403 2024-09-16T05:09:25.186 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-16T05:09:25.186 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T05:09:25.196 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T05:09:25.196 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T05:09:25.210 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T05:09:25.220 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T05:09:25.229 INFO:teuthology.task.internal:Duration was 1914.856427 seconds 2024-09-16T05:09:25.229 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T05:09:25.239 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T05:09:25.239 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T05:09:25.421 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T05:09:25.421 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-09-16T05:09:25.422 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T05:09:25.433 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T05:09:25.433 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T05:09:25.508 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T05:09:25.508 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T05:09:25.689 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T05:09:25.699 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T05:09:25.700 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T05:09:25.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T05:09:25.735 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T05:09:25.804 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-09-16T05:09:25.827 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T05:09:25.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T05:09:25.873 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T05:09:25.882 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T05:09:25.883 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:00:02-rados-squid-distro-default-smithi/7906051/remote/smithi012 2024-09-16T05:09:25.884 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T05:09:25.938 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T05:09:25.938 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T05:09:25.982 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T05:09:25.998 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T05:09:25.998 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T05:09:26.062 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T05:09:26.062 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T05:09:26.069 INFO:teuthology.orchestra.run.smithi012.stdout: 658464 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 05:09 /home/ubuntu/cephtest 2024-09-16T05:09:26.070 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T05:09:26.081 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T05:09:26.111 INFO:teuthology.run:Summary data: description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest}} duration: 1914.856427192688 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T05:09:26.111 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T05:09:26.200 INFO:teuthology.run:pass